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EVALUATING JOB TRAINING PROGRAMS IN THE UNITED STATES: 
EVIDENCE AND EXPLANATIONS 

W. Norton Grubb 

Over the past thirty years, the institutions in the United States that educate and train 
people for employment have grown in number and complexity. High schools, the 
traditional locus of vocational education, still provide some job-specific education, but 
increasingly vocational education takes place in postsecondary institutions including area 
vocational schools, community colleges, and technical institutes. The development of job 
training programs, first through manpower programs during the 1960s and then in 
Comprehensive Employment and Training Act (CETA) during the 1970s and the Job 
Training Partnership Act (JTPA) during the 1980s, added to the number of programs, as 
have job training programs provided through the welfare system, especially the Job 
Opportunities and Basic Skills Training (JOBS) program of the Family Support Act of 
1988. Other special-purpose programs have proliferated, included those for dislocated 
workers — individuals who become unemployed as a result of economic dislocations 
beyond their control, like the decline of defense industries or competition from foreign 
producers. Many states have initiated their own economic development programs, 
providing yet other training resources intended to lure employment from other areas, 
facilitate local expansion, or forestall employers from leaving the area. Proprietary schools 
have also increased their enrollments, partly in response to increased student aid during the 
1970s Thus the "system" of work-related education and training institutions — those that 
consciously prepare individuals for relatively specific occupations that do not require a 
Bachelor's degree — has become increasingly complex and variegated. 



In large part, the expansion of job training programs has reflected a concern with 
particular economic problems, especially those of unemployment, underemployment, and 
poverty. Job training programs were first discussed as a response to the unemployment 
created by the 1960-61 recession. Since then, periods of recession and unemployment, and 
specific kinds of unemployment (like the increases in dislocated workers unemployed as a 
result of substantial economic changes) have generated interest in job training programs as 
potential solutions. And the expansion of welfare programs supporting low-income 
families during the 1960s generated the realization that poverty was not likely to wither 
away of its own accord, and one solution has been to propose job training. Of course, the 
problems of poverty, unemployment, and underemployment are closely related, and so — 
despite a proliferation of programs for specific purposes — it is not surprising to see an 
overall pattern to the job training programs in the United States. 

In the process, an important distinction has emerged between education and job 
training. The difference is not always clear, since some short-term, job-specific education 
programs look quite similar to job training. However, there are at least six differences 
between the two. First, job training programs are generally much shorter. Many of them 
last 10 - 15 weeks, with part-day attendance, so that the number of contact hours may be as 
low as 40 hours; the average program length in the recent JTPA evaluation was 3.5 months 
(Orr et al., 1994, Exhibit 3.18). In contrast, the shortest common postsecondary education 
programs — for occupational certificates — generally last two semesters (or aboi 1 30 
weeks) of full-time enrollment, involving 360 to 1,000 contact hours; and two-year 
Associate programs dominate the programs of community colleges. 1 

Second, education programs — particularly in community colleges and other two- 
year colleges, and area vocational schools — are generally open to all members of the 
population; but job training programs are open only to those who are eligible — for 

1 Of course, student attendance patterns can reduce these longer education programs to individual courses. 
Area vocational schools often offer short programs of part-time attendance over one semester of about 15 
weeks, and are therefore no more intensive than many job training programs. 



example, the long-term unemployed or dislocated workers in JTPA, or welfare recipients in 
welfare-to-work programs. (The issue of eligibility reflects the origin of job training as a 
solution to particular economic problems: only those who have suffered from these 
problems are eligible, not the population as a whole.) By construction, then, job training 
programs enroll individuals who have had particular problems in employment; while some 
problems may be due to overall employment conditions, others may be due to deficient 
skills, behavioral problems, and other personal traits. 

Third, most education programs take place in educational institutions that are well- 
institutionalized and standardized — in high schools, community colleges, and four-year 
colleges. In contrast, job training services are offered in a bewildering variety of 
educational institutions, community-based organizations (CBOs), 2 firms, unions, and 
proprietary schools, making it difficult to determine how services are organized and 
provided. 

Fourth, the kinds of services provided in education programs are relatively 
standard: for the most part they offer classroom instruction, with both academic and 
vocational courses, often including labs, workshops and other "hands on 1 ' activities. Job 
training programs offer classroom instruction too, in both basic (or remedial) academic 
subjects like reading, writing, and math, as well as in vocational skills; but they also offer 
on-the-job training 3 , where individuals are placed in work sites, presumably to learn on the 
job; work experience, where individuals work for short periods of time; job search 
assistance, in which clients are given some training in how to look for work, write 
resumes, file job applications, interview for jobs, and the like; and job clubs, in which 
clients are required to spend a certain amount of time looking and applying for jobs. Some 

2 CBOs are private organizations, incorporated under state laws, that provide many different kinds of social 
and educational services. Some of them are identified with groups of individuals — e.g., they represent the 
black community, or Hispanic migrant workers, or the disabled, or older women returning to the labor 
force; some of them represent particular services, like child care or homemakers services for the elderly. In 
general, there is minimal government regulation of CBOs, and their quality varies greatly. 

3 One of the issues here is whether there is any training involved in on-the job training, or whether it is the 
same as short-term work experience; see the discussion in Section IV. 3 below. For evidence thai much on- 
the-job "training" does not provide training, .»e Kogan et al. (1989). 



programs provide counseling as well, to give clients both information about labor market 
opportunities and "life skills" like the abilities to plan. Job training programs also support 
placement efforts somewhat more often than educational institutions do, reflecting another 
division between education and job training: those in educational institutions are likely to 
declare that they are responsible for "education, not employment", while those in job 
training are more likely to accept that they have a responsibility for placing individuals as 
well as training them appropriately, Unfortunately, the variety of such services is so great, 
and the forms they take is so varied, that it is difficult to know precisely what takes place. 
As a result the evaluations of specific program components (reviewed in Section JH.5) is 
not particularly comprehensive and is generally inconclusive. 

Fifth, the goals of education programs are typically quite broad, and generally 
encompass political, moral, and intellectual purposes as well as occupational ends; but job 
training programs are focused exclusively on preparing individuals to become employed. In 
the case of welfare-to-work programs, the single goal is to get welfare recipients employed 
as quickly as possible so they can move off the welfare rolls. Because the goal of job 
training programs is so unambiguous, and because there are no intrinsic benefits to job 
training — no one would declare, that being in a job training program is fun, or a social 
activity, or a normal part of growing up, as Americans might say about schools and 
colleges — there has been a long history of evaluating them to ascertain their effectiveness. 
These evaluations have also become increasingly sophisticated over time — certainly much 
more sophisticated than those of education programs. The results have also influenced 
public policy in a way that has not been true in the education system because the political 
pressures in education — the support of parents for programs that benefit their own 
children, including such diverse offerings as those aimed at low-income students, or 
limited-English-speaking (LEP) students, or gifted students, for example — are generally 
lacking in job training programs, where the only justification for public support is their 
reduce of unemployment, poverty, and the receipt of welfare. 
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Finally, job training programs differ from education programs in constituting a 
separate kind of system, a "second chance" system in some ways parallel to but 
disconnected from the "first chance" educational system. Over the course of 150 years, the 
education system in the United States has developed a well-articulated series of offerings 
from kindergarten (now often extended to pre-school programs) to the university level. But 
for those who have left this system without adequate skills, the job training system can be 
interpreted as a second chance to get back into the mainstream of the labor force. In 
general, the establishment of this second chance system is one manifestation of a generous 
American impulse: to provide opportunities to individuals through various forms of 
learning, and to be inclusive of all who might benefit from such activities. However, this 
second-chance system is much younger than the education system; it spends less, is more 
disorganized, has lower status, and is poorly institutionalized so that it cannot resist purely 
political pressures. As a result it has been subject to revision by nearly every President so 
that it lacks the stability of the education system. Given these differences, it is not 
surprising to find that the job training system is not especially effective — partly, as I will 
argue in sections IV and V, because of the separation of education and job training in 
separate arenas, a division that has been detrimental to both of them. 

In this monograph I review the effectiveness of job training programs in the United 
States, concentrating on the most recent and most sophisticated evaluations. 4 Section I 
describes in greater detail the variety or job training programs described, explaining why it 
is appropriate to consider a wide variety of programs. Section II outlines the preferred 
methodology of recent evaluations using random assignment methods, and clarifies both 
the strengths of this approach and its inevitable weaknesses. Section IE then presents a 
series of results, first for job training programs (ELI), then for welfare-to-work programs 

4 This monograph is not a formal meta-analysis, which would require statistical analysis of outcome 
results, suitably standardized, from a large number of studies. A formal meta-analysis has been carried out 
by Fischer (1995), and will be cited as appropriate; however, like most meta-analyses, the summarization of 
evidence makes it difficult to understand what might have caused the outcomes. Instead, I take the approach 
of presenting results from a number of specific evaluations in a series of tables illustrating the kinds of 
findings that have been typical. 



(m.2), and finally for special experimental programs (m.3). These results are followed by 
the outcomes for different population groups (HI.4), for different types of services (HI.5), 
for the effects of programs over time (m.6), and for different programs (IEL7). Finally, in 
Section EH.8, 1 present some recent benefit cost-analyses. A series of tables drawn from the 
evaluations accompanies this section, so that the reader can see concretely the results of the 
major evaluations. 

The major question these evaluation address, of course, is whether job training 
programs been successes or failures. A conventional reading of the evaluations is that many 
(though not all ) job training programs lead to small but statistically significant increases in 
employment and earnings, and (for welfare recipients) small decreases in welfare 
payments; where cost-benefit analyses have been done, the social benefits usually outweigh 
the costs (but not always). One might conclude that these programs have been successful 
and should be continued. However, the gains in employment and earnings are, from a 
practical s andpoint, quite small: they are insufficient to move individuals out of poverty, or 
off of welfare; their effects very often decay over time, so that their benefits are short-lived; 
and as they are currently constructed they certainly do not give individuals a chance at a 
middle-class occupation or income. In my interpretation, therefore, the successes of job 
training programs have been quite modest, even trivial — and that dismal conclusion 
requires some understanding of why that might be true. In Section IV, therefore, I present 
a series of possible explanations for the weak results of job training programs. The reasons 
for failure are necessarily more speculative than are the outcome results in Section m, 
which are based on harder data and (in many cases) random assignment methods; but some 
understanding of why job training programs have had such modest results is necessary to 
develop recommendations that could remedy such programs through public policy, or 
create more effective programs from the start. 

Although the benefits of current job training programs have been small, they 
problems they address — unemployment, underemployment, and welfare dependency 
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are too serious to ignore. Therefore, rather than abandoning job training, the appropriate 
response is to determine how to reform them. The conclusion therefore presents a vision of 
how job training programs could be structured in ways that avoid the reasons for failure 
outlined in Section IV. This vision has in fact been embodied in current federal legislation 
— the School -to-Work Opportunities Act, passed in May 1994 — that applies to high 
schools and community colleges. But its implications for job training programs have not 
yet been developed, and so the purpose of the conclusion is to clarify how reforms now 
proposed for the education system might benefit job training programs as well. In the end 
such reforms could eliminate the unproductive division between education and job training 
that has developed over the past thirty years. 
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I. The Nature of Job Training Programs Reviewed 



Several different strands of development have created the current "system" of job 
training programs — in reality, a non-system with a bewildering variety of purposes, 
services, and funding. One strand began with manpower programs established in 1962, in 
response to the recession of 1960-61. The Manpower Development Training Act (MDTA) 
of 1962 established job training programs administered by the Department of Labor, 
separate from federal support of vocational education. 5 The establishment of an 
independent funding mechanism for programs outside the schools was based in part on the 
poor reputation of vocational education and in part on a general feeling that secondary 
schools were not equipped to provide nontraditional training for adults. These programs 
were then consolidated in the Comprehensive Employment and Training Act (CETA) of 
1973. Like the programs it combined, CETA provided little role for state governments and 
gave local administrative units greater decision-making power over the types of training 
provided, the groups of individuals served, and the institutions offering training and other 
services. 

States were given additional authority by JTPA, the successor to CETA, enacted in 
1983. State governments now designate local service delivery areas (SDAs), and they can 
establish priorities for SDA use of a portion of the federal grant. However, JTPA still 
remains a federal rather than a state program: Nearly all funding comes from the federal 
government, federal regulations apply to all programs nationwide, and many states have 
made no effort to assert a role in policy-making beyond that required by federal regulation. 
The development of job training programs did more than simply add new funding sources 
for work-related training; it also dramatically changed the types of institutions that provide 
training. Job training programs since the 1960s have been characterized by their use of 
CBOs, unions, private firms, and other institutions — private alternatives to conventional 

5 On the history of manpower policy, see National Academy of Sciences (1975). 
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high schools, community colleges, and technical colleges — to provide training and related 
services. This aspect of joL> training has given the education and training system greater 
variety and fluidity and has helped erode the boundary between public and private 
programs. 

Job training also marked a turn toward private sector participation in public 
programs, not only with the funding of private organizations such as CBOs, but also with 
the establishment of Private Industry Councils (PICs), at least 5 1 percent of whose 
members must represent the private sector. The PICs are responsible for policy guidance 
and program oversight, and they must approve SDA training plans. They also have the 
option to administer JTPA programs directly, although fewer than 20 percent do. It is 
important to note that "private sector representation" refers to employers, and not 
representatives of labor like unions. Overall, the participation of unions in the job training 
system is relatively weak, partly because unions now represent only 1 1 percent of 
employees in the United States. 

JTPA also differs from earlier programs in the nature of the mandates it imposes. 
While its predecessors focused on the types of services that local agencies could deliver, 
JTPA emphasizes outcomes by requiring that SDAs meet specific performance standards. 
The federal government has identified twelve standards 6 from which states select eight that 
SDAs must meet; states may add standards of their own and may use either a federal 
adjustment model or one of their own design to take into account the demographic and 
labor market characteristics of individual SDAs. In tneory, the imposition of performance 
standards is a way of making JTPA more effective. In practice, however, performance 
standards can be manipulated by local programs, and they prove to be uncorrelated with the 
effects measured by more sophisticated evaluation techniques (Doolittle et al,, 1993, p, 10); 

6 For program years 1988 and 1989, the adult standards included percent of participants placed in jobs (68 
percent); average hourly wage at job placement ($4.95); average cost of placement ($4,500); percent of 
welfare recipients placed (56 percent); percent of participants employed at 13- week follow-up (50 percent); 
number of weeks worked at follow-up (8); and weekly earnings at follow-up ($177). The youth standards 
included a positive termination rate, cost per positive termination, entered employment rate, and 
employability enhancement rate. 



they have made local programs concerned with the details of performance measures, but 
not with effectiveness in a broader sense. 

A second strand of development has concentrated on welfare recipients. 
Historically, welfare in this country — principally, the program known as Aid to Families 
with Dependent Children (AFDC) 7 — was provided only to mothers with children so they 
could stay at home and care for their young. However, as working has become 
increasingly common for all women in the United States, including mothers of young 
children, there has been increasing pressure to get welfare mothers (and fathers 8 ) into 
employment and off the welfare rolls. The first efforts were established in 1962 in the 
Community Work and Training Program. Like MDTA, it provided funds from the 
Department of Labor, which could be used by welfare programs at the local level, 
bypassing the vocational education system. The Economic Opportunity Act of 1964 
included yet another program designed to encourage work, the Work Experience and 
Training Program. In 1967, as part of the far-reaching Amendments to the Social Security 
Act, the Work Incentive (WIN) program was established as a voluntary work program. 
Although WIN was made mandatory for welfare recipients in 1971, it was not funded at a 
level that made widespread participation enforceable, and therefore it remained a limited and 
voluntary program. 

Another strand of the Lyndon Johnson's War on Poverty was the "services 
strategy" developed as an antidote to poverty. This strategy provided a variety of support 
services (such as child care and transportation) to enable welfare recipients to work their 
way off welfare. As embodied in the 1967 Amendments to the Social Security Act, it 
included funding for short-term training. The support services, which were then 

There are many other forms of aid to low-income families that also constitute welfare, including food 
stamps, housing subsidies, child care and other social services; and many other programs for individuals 
with low earnings should also be considered welfare, including Social Security for the elderly. However, the 
political concern with "welfare" and efforts to reduce welfare costs almost always concentrate on AFDC first 
and foremost. 

0 AFDC was extended to two-parents families in 1967, because of the understanding that confining welfare 
only to mothers with young children would cause some fathers to abandon their families. 
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consolidated in the Title XX Amendment of 1973, provided funds for social services to 
states and gave states greater authority to decide which services should be provided. Title 
XX emerged largely intact (though with considerably reduced funding) in the Social 
Services Block Grant, enacted in 1981. However, in practice, work-related services 
(including training) were rarely provided under Title XX, which focused instead on 
rehabilitating families on welfare and preventing abuse, rather than facilitating employment 
(Dickinson, 1986). 

In 1981 , the Reagan administration, building on a history of "welfare-to-work" 
programs that forced welfare recipients to work in exchange for grants, allowed states to 
develop their own programs for getting welfare recipients back to work. The state 
programs that developed were, not surprisingly, enormously varied. Most relied heavily 
on job search (i.e., short-term assistance in applying for work, but no other training or 
ii pport services) and work experience or OJT, both accomplished through short-term job 
placements. A few developed Community Work Experience Programs (CWEPs), in which 
welfare recipients provide community service in amounts related to the size of their 
grants — equivalent to the traditional conception of "workfare." Although 84 percent of the 
programs offered vocational skills training and 72 percent provided post-high school 
education, in practice only 2.3 percent of the welfare recipients participating in these 
programs received any skill training, and only 1.6 percent enrolled in postsecondary 
education (most of these were in Massachusetts, Michigan, and California). In fact, only 
3.2 percent received remedial education; even the most basic forms of education and 
training were quite rare. 9 In practice, then, experimentation with various kinds of services 
and "welfare-to-work" 10 strategies led to an emphasis on job search, rather than education, 
training, or other services. 

9 These figures refer to participation in WIN demonstration programs (U.S. GAO, 1987). 
10 Applying the term "workfare" to the experimental welfare programs of the 1980s is somewhat misleading 
because few of them included CWEP, and the mandatory elements that were historically part of workfare— 
e.g., the threat that welfare recipients would lose their grants if they failed to comply with work 
requirements— have not been frequently used. Enrollments in these new programs have been kept relatively 



The most recent development in this area is the Family Support Act of 1988, which 
requires all states to establish JOBS programs to increase the employment of welfare recipi- 
ents. The legislation provides federal matching funds — ranging from 50 percent to 72 per- 
cent of total costs — for a variety of work-related services, including job search, work 
experience, counseling, child care, and other support services, and all forms of remedial 
education, vocational education, and training. This new legislation combines the services 
strategy of the 1960s with the work-related emphasis of WIN (including the use of 
education and training). 

With JOBS, Congress crafted a program that combines a mandate (participation is 
mandatory for all AFDC recipients who are single heads-of-household and who have no 
children under three years of age 11 ), inducements in the form of services (e.g., one year of 
health care after participants obtain a job, and transitional child care) to reduce the cost of 
moving from welfare to employment, and capacity-building through longer-term 
investments in education and training (typically up to two years). As it has done with 
vocational education and JTPA, the federal government has attempted to target JOBS 
service recipients. To avoid a situation in which states primarily serve those most likely to 
get off welfare even without additional assistance, the federal government requires that 55 
percent of states' JOBS funds be spent on those most at risk of long-term welfare 
dependency (e.g., young mothers who are also high school dropouts). 
To some extent, the federal government has specified the services that can be provided by 
designating which ones are reimbursable. It has also been fairly specific in defining service 
levels. For example, the proposed JOBS regulations specified that only people who were 
spending at least 20 hours a week in authorized activities could be counted by states as 

low, partly for reasons of cost, so there has been greater emphasis on voluntary than on coerced 
participation. 

1 ] ln a departure from past welfare policy, JOBS also allows families in which both parents arc present in 
the home and the primary wage-earner is unemployed to qualify for benefits and services, and it requires that 
one parent participate in training. However, like WIN, funding limits mean that JOBS will not function as 
an enforceable mandate. The federal government is requiring that states enroll 7 percent of their total 
welfare caseload in 1990 and that the proportion increase to 20 percent by 1995. 

12 Itf 



JOBS participants. The states protested vigorously, arguing that some effective education 
and training programs require fewer than 20 hours a week (e.g., a full community college 
courseload typically includes 12 to 15 hours of classroom work per week). The U.S. 
Department of Health and Human Services (HHS) subsequently changed the regulations to 
make the 20-hour requirement the average for groups of participants (Kosterlitz, 1989). 
Despite clear federal specifications for some aspects of service delivery, the federal gov- 
ernment has given the states considerable flexibility in deciding which types of services will 
be provided, who will provide those services, and the scope of the programs. Even though 
the JOBS program represents a major new source of federal funding for education, job 
training, and related services, it requires a substantial matching of funds (between 37 and 
50 percent, depending on the services) from the states. It appears that although most states 
will eventually increase their total spending for welfare recipients, the level will vary 
greatly — thus perpetuating the differences in welfare benefits and services among states. 
Many state legislatures will appropriate insufficient funds to match the maximum federal 
funding. 

The question of what program outcomes the federal government expects is still 
open to argument. HHS will not establish performance standards until later in 1995. In 
the meantime, debate will continue over what many have called the "hamburger flipper" 
question: Is it better to move welfare clients quickly into jobs, even if those jobs do not 
pay enough to lift them out of poverty, or is it better to spend more time and money to 
increase the clients' chances for longer-term, higher-paying employment that may keep 
them from falling back onto welfare in the future? (Kosterlitz, 1989). The way the answer 
to this question is articulated in the federal performance standards will largely shape the 
amount and type of work-related education and training available to welfare recipients. 

Still other job training programs have been developed in response to other more 
specific problems. For example, a number of programs provide assistance specifically for 
veterans, since disabled veterans in particular often have a difficult time entering the labor 
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force; dislocated worker programs have expanded because of the special need for 
experienced workers laid off through no fault of their own (e.g., because of the decline of 
the timber industry in some states, the decline of defense and aerospace in other states, the 
possibility of unemployment because of NAFTA, particularly for border states) to be 
retrained for other kinds of employment; and vocational rehabilitation programs have 
focused on the special employment problems of disabled individuals. There has been a 
strong tendency for Congress to respond to specific new problems with a specific new 
program, rather than to incorporate new purposes into old programs; this in turn has 
generated a proliferation of job training programs with roughly the same goals — the 
enhancement of employment — but for different groups with different kinds of barriers to 
employment. 

A particular concern has been the development of programs for youth. The problem 
of high school dropouts is an old one, dating almost to the turn of the century, but it has 
become increasingly serious as the employment prospects of dropouts have become worse 
relative to high school graduates. In addition, programs for youth have the special aura of 
prevention: if they can steer individuals away from unemployment, and away from crime, 
drugs, and (for girls) early pregnancy, then they can prevent social costly problems in the 

future. In response, a large number of programs focused on the employment of youth 

both those who have dropped out of high school, and those still in school but considered 
likely to drop out — have been developed, particularly within CETA and then JTPA, and 
their effectiveness is a subject of special concern. Some of these programs are properly 
considered job training programs — that is, they offer education and training to equip 
young people with new skills — while others are really work experience programs, 
particularly the summer youth programs that employ some young people during the 
summer. A positive view is that work experience itself will teach young people the personal 
attributes necessary for stable employment; a more general perspective is that such efforts 
are palliatives to keep troublesome youth off the streets. In this monograph I consider the 
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programs targeted for youth in JTPA (Section EDL1) and in experimental programs (Section 
n.3), and in evaluating the effects on particular populations (Section IH4). I then interpret 
the especially dismal results for youth in Section IV.9. 

The results of these many strands of development is that a bewildering array of job 
training programs exist. Indeed, when the General Accounting Office (GAO) examined 
federally-funded employment and training programs, they counted 163 programs spending 
$20.4 billion in 1995 (U.S. G.A.O. 1994). 12 These counts are somewhat misleading, 
since they disaggregate different titles of certain programs (for example, JTPA is counted 
as 20 different programs because of its different sections), they include education programs 
(like literacy efforts and student grants and loans) that are only distantly related to job 
preparation, and they include other programs (like Community Development Block Grants) 
that fund a variety of services other than job training. Still, the count is instructive because 
it illustrates nicely the proliferation of purposes and the widespread perception of how 
incoherent and fragmented the education and training "system" has become. For purposes 
of understanding the rough magnitudes of different programs, Table 1 summarizes the 
GAO figures about education and job training programs. Clearly the major job training 
programs are JTPA and JOBS, and they are larger than any other federal programs with the 
exception of students grants and loans for postsecondary education. 13 

Finally, there have been a large number of experimental programs, often started by 
private foundations in order to test particular approaches to enhancing employment. Some 
of these efforts have been particularly intensive, and others have concentrated on providing 
services of particularly high quality. They therefore might provide information about what 
excellent job training programs — well-designed, independent of political manipulation, 
and freed of having to operate under normal pressures — might accomplish. Several of 

ljL See also the massive review of federal programs compiled by the National Commission for Employment 
Policy (1995). F J 

Although about one quarter of student aid goes to proprietary schools, most of the remainder goes to 
students in four-year colleges and universities, not to students in occupationally-spccific programs in public 
community colleges and technical institutes. Student aid therefore provides oniy limited support for job 
training. 
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these experimental programs have been carefully evaluated, and I present their results in 
Section m-3 — though with the caveat that such programs may be quite different than 
those operated with public funding. 14 

There are several reasons for examining the evaluations of a variety of job training 
programs, rather than confining a review like this to the main job programs of JTPA. One 
is that there is considerable overlap among these programs because of the ways they have 
been established. JOBS programs for welfare recipients often send their clients to local 
JTPA programs — sometimes on their own initiative, and sometimes at state directive — 
and so the two programs use the same services and providers. Both JTPA and JOBS may 
send their clients to vocational education in community colleges, areas vocational schools, 
and technical institutes — particularly when clients are able to find their own education and 
training arrangements (known as "individual referral") or when there is a fiscal incentive to 
do so. 15 Secondly, the kinds of services various job training programs provide overlap 
considerably, so that information from one program is useful in judging the effectiveness 
of others. Finally, the discussions of job training programs in the United States generally 
commingle information about different types of programs, and so it is necessary to 
understand the evaluations of all these programs. 



14 See also the distinction made by Gueron and Pauly (1991) between "broad coverage" programs, which 
include all the elements of a complete job training program and are designed to reach a broad range of 
individuals (and where participation is often mandated for welfare recipients), and "selective-voluntary" 
programs, for which program administrators can select who can enroll or individuals can volunteer to 
participate or not. The experimental programs examined in Section III.3 are examples of "selective- 
voluntary" programs. 

15 In many places, states provide funding per student to individuals in community colleges and technical 
institutes, so that if JTPA or JOBS sends an individual to a community college state aid to the college — 
and not JTPA or JOBS funds — pays most of the costs of the programs, making this kind of referral 
cheaper than paying the full costs of the program in a CBO. This kind of "cost-shifting" leads to individuals 
being funded out of several public sources simultaneously: an individual in a community college could 
receive subsidy through state aid to the college, federal grants and loans through student aid, and subsidy 
through JTPA. or JOBS. See Grubb and McDonnell (1991). 



II. The Methodology of Job Training Evaluations 



The first round of job training programs — those begun in the 1960s under MDTA 
— were evaluated with less sophisticated methods than have now become standard. (See 
Table 2 and the associated discussion in Section LI.) Effectiveness was very much an 
issue when job training programs were consolidated in CETA, in 1973 — because of 
suspicions that may of the manpower training programs of the 1960s were not especially 
effective. The CETA program therefore generated a huge amount of evaluation, taking up 
literally several meters of shelf space (summarized in Taggart, 1981). The quantitative 
evaluations of CETA used quasi-experimental methods, generally comparing CETA 
enrollees and control groups with similar characteristics (gender, race, education, labor 
market experience, and the like) and comparing earnings of experimental groups and 
control groups using regression methods. 16 In addition, a number of studies administered 
surveys to CETA trainees about their experiences; others described the nature of training in 
great detail; and still others used what might be called ethnographic techniques, observing 
individual programs closely to see what was going on. These results generated information 
on employment outcomes, to be sure — summarized in Section m-A below — but they 
also described many other dimensions of programs. 

Beginning in the 1980s, outcome evaluations began to use true experimental 
methods for evaluation: individuals are recruited for programs, and a random sample (the 
"experimentals") are allowed to enroll in the program while the others (the "controls") are 
administered questionnaires to collect roughly the same information as the experimentals 
about the services they receive and their employment history. The ethical dilemmas 
involved in experimental methods have been avoided by using volunteers and by recruiting 
more individuals than can be accommodated in the program to be evaluated, so that one 

16 CETA generated a data set — the Continuous Longitudinal Manpower Survey (CMLS) — that followed 
a several waves of CETA clients and also contained information about control groups, and CLMS was used 
for many of the evaluations; for a survey, see Bamow (1986). 
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could argue that some individuals would not be served even if the programs were not being 
evaluated using an experimental design. In addition, the effectiveness of these programs is 
genuinely unknown so that — unlike denying an individual access to a vaccine known to 
work against a particular disease — no one is being kept out of a program that would 
surely increase their life chances. 

The great advantage of experimental methods is that they can eliminate the 
possibility that various factors unconnected to program effectiveness are responsible for 
any findings. In job training programs, there are three such factors that are particularly 
dangerous: selection effects, maturation effects, and regression to the mean. Selection 
effects operate because job training programs by construction select those individuals who 
have certain barriers to employment — low education levels, little work history, perhaps 
motivational problems, or histories of drug and alcohol abuse — and therefore might be 
expected to benefit least from any training program; the variety of these characteristics is so 
great, and so unmeasurable, that it is difficult to create an equivalent control group without 
experimental methods. However, these negative selection effects are complicated by other 
selection effects created by the administration of programs. In order to look good, job 
training programs have an incentive to choose the most able and job-ready of the 
individuals who are eligible — a process known as "creaming". This creates a positive 
selection effect in addition to the negative selection effect involved in eligibility for the 
program. Moreover, this kind of effect may operate differently over the business cycle: 
when unemployment falls, the most job-ready individuals are able to find jobs so that 
programs have to work harder to recruit people to enroll — and may have to enroll the least 
job-ready individuals with multiple employment problems. Paradoxically, then, in boom 
times when low unemployment makes placements somewhat easier, the individuals 
enrolled are the least job-ready; when unemployment is high and placements more difficult, 
the most job-ready individuals are likely to be enrolled because of "creaming". It would be 
virtually impossible, then, to construct a control group that is co parable to those enrolled 
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in a job training programs except under experimental conditions, since there are too many 
administrative, economic, and personal factors that affect the composition of a job training 
program. 

Maturation effects occur when individuals improve their conditions by aging or 
maturing. This is a particularly likely result for youth, who suffer much higher rates of 
unemployment and lower earnings when they are young and then gradually mature into the 
more stable employment and earnings patterns of adults — most of them without the help 
of any particular program. Maturation effects are also likely for measures of academic 
achievement, for knowledge about the labor market, for risk-taking behavior, and for 
certain measures of disruptive behavior including drug use and criminal activity. Without 
considering this phenomenon, youth programs may look effective over time as those who 
have enrolled in them mature, even though the program may have had no effect on this 
process. 

Regression to the mean is another problem. By construction, job training programs 
enroll individuals who have had problems in employment. But some of these individuals 
may have had an unlucky spell — an unexpected layoff, for example, for an individual 
with adequate job skills in what is otherwise a healthy local economy — and can be 
expected to find employment on their own within a few months (that is, they regress back 
to their normal conditions of employment after a while). For such individuals a job training 
program might speed up the return to employment, but may not make any difference to 
whether such an individual finds employment again — in contrast to an individual who 
lacks fundamental job skills, who is unlikely to find employment without training. The 
problem of regression to the mean is a particularly serious problem in welfare programs: a 
large fraction of the welfare population is on welfare for a brief period — following a 
layoff, the departure of a wage-earning family member like a husband, a medical 
emergency — but then finds employment and leaves the welfare rolls after a short period of 
time. If large numbers of these "temporary" welfare recipients are enrolled in job training 
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programs, then it will appear to be a success — though all that may be happening is that 
normal turnover rather than the effectiveness of the job training program is causing some 
individuals to find employment and leave welfare. In the quasi-experimental evaluations of 
CETA programs in the late 1970s, the apparently greater increase in earnings for 
experimental groups compared to comparison groups turned out to be due to regression to 
the mean for males, though for females job training programs increased earnings by 
slightly more than would be expected from such a pattern (U.S.C.B.O. , 1982, and Figure 
1). 

However, with experimental methods every kind of selection effect is eliminated, as 
are maturation effects and regression to the mean 17 — so that any differences in 
employment after a job training program can be attributed to the program rather than to 
other causes. Despite these advantages of experimental methods, however, there remain a 
number of disadvantages, or evaluation problems which the use of experimental methods 
has not always been able to resolve: 

• Experimental methods sometimes treat the program as a black box, or a treatment 
whose nature is unknown. Learning that a program increases earnings may not be 
especially valuable in determining which elements of a successful program make the 
difference, and which need to be incorporated in any replications. While more recent 
evaluations have become somewhat better about analyzing the effects of particular services, 
there is still a tendency to treat evaluation as a statistical problem — ascertaining whether a 
variable Y (like earnings) is caused by a variable X (like program participation) — rather 
than a programmatic problem of determining what kinds of services, offered to which 
individuals, reduce unemployment, poverty, and welfare. 
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There can be. of course, differences between experimental and control groups due to sampling error 
Therefore most evaluations use regression methods to control for the effects of various personal 

2K' Sf ^ bCtWeen ex P erimcntal and contro > g™"Ps ~ but with the knowledge that 
variables describing program participate are uncorrected with background variables or unmeasured 
charactensUcs like moUvation. 
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• Even the best-designed experiments can degrade into quasi-experiments because 
of the non-random behavior of experimental groups. For example, in the most recent JTPA 
evaluations, experimental groups were chosen to enroll in job training; but only a fraction 
actually did so since some of those assigned to job training (the "assignees") lost their 
motivation, or moved away, or otherwise did not enroll in the program. Those who fail to 
show up are likely to be the least motivated, the least organized, and the most plagued by 
various problems that affect employment. As a result the average increase in earnings 
among assignees is lower than it is among those who enroll in the programs — and there 
are disagreements about whether the benefit per assignee or per enrollee ought to be used in 
deciding whether the program works. 18 Similarly, attrition in the course of job training 
programs is non-random, since those who complete programs are likely to be more able 
and motivated. And in the evaluation of Florida's Project Independence, some members of 
the control group — which was supposed to have no access to special services — by 
mistake were allowed to attend an orientation session and to participate in some Project 
Independence activities, "contaminating" the difference between the treatment group and the 
control group (Kemple, Friedlander, and Fellerath, 1995). Thus even well-designed 
experiments can turn into quasi-experiments, with some of the most interesting results 
affected by non -random selection. 

• An evaluation which assigns volunteers randomly to as job training program or to 
a control group may not use random assignment for other issues of interest. For example, 
job training typically uses a mix of classroom instruction in vocational skills, basic 
academic skills instruction (or remediation), on-the-job training or work experience, and 
job search assistance — but even though volunteers may be assigned randomly to a 
program, they are typically not assigned randomly to the services within that program. If 
programs match individuals to services, then (for example) the least job-ready individuals 

18 A contrary argument could be made: that assignees who don't enroll find a job in the interim, and are 
therefore more job-ready and motivated. However, if this were true, then the benefit per assignee would be 
higher than the benefit per enrollee, contrary to the evidence. 



may be assigned to longer term basic skills instruction, while the most job-ready 
individuals are given job search assistance — potentially making job search assistance look 
effective while basic instruction looks ineffective. Under these conditions the effects of 
being in a program are evaluated experimentally, but the effects of different services can be 
evaluated only with quasi-experimental methods — and therefore some evaluations have 
failed to conclude anything about the effects of different services. In addition, sample sizes 
also affect any conclusions about specific services: if the numbers receiving any particular 
service are small, then conclusions about effectiveness are difficult to make within the 
normal limits of statistical significance. 

The problems affecting the evaluation of different services also influence the 
evaluation of effectiveness for different subgroups within the population — for example, 
women compared with men, high school graduates compared with drop-outs, those with 
more labor market experience compared to individuals with little experience. But of course 
it is possible that job training programs are effective for certain sub-groups but not others 
(as reviewed in Section D3.4 below) — including the important question of whether job 
training is most effective for the least job-ready individuals (in which case "creaming" 
should definitely be avoided), or whether it is most effective for the most job-ready. 

Finally, the problems in evaluating effectiveness for different services and sub- 
groups extend to the evaluation of particular programs as well. That is, a national job 
training programs like JTPA is in reality an agglomeration of over 500 programs, each 
administered locally — and any average effect masks the distribution around this average 
caused by the existence of highly effective programs simultaneously with truly dreadful 
local efforts. The most effective programs may be the most valuable guides to improving 
practice, of course, so — if they can be identified — their characteristics may provide the 
best information about how to improve programs. While the early evaluations did not 
address the effectiveness of individual programs, some of the most recent evaluations have 



22 2 * 



managed to detect individual programs that are more effective than the average (reviewed in 
Section IIL7 below). 

• Evaluations that focus on individuals in job training programs can determine 
whether the likelihood of employment is greater for the enrolled group compared to a 
control group; but they cannot tell whether the aggregate employment in a region increases. 
It is possible that the employment of those in job training programs comes at the expense of 
other individuals who would have gotten those jobs had there been no job training 
program; that is, a job training program merely displaces some potential workers with 
others. 19 This is particularly likely to be the case where a "training" program establishes 
strong relationships with employers, in effect becoming a preferred source of moderately 
skilled labor by serving as a screening mechanism for motivation, stability, or other traits 
that particular employers want — but without providing much training. One could argue 
that such displacement effects will be captured by the nature of the control group, which by 
construction includes those who might otherwise have gotten the jobs that trainees receive; 
but given the small size of control groups relative to the pool of potential workers, this 
seems unlikely. To my knowledge no study has examined the potential extent of such 
displacement effects. 

The danger of displacement also reflects a difference between human capital models 
of earnings and employment — in which education and training instill new competencies 
that increase the productivity and then the wages and earnings of individuals — from 
screening and signaling models, in which education or job training signals the greater 
competencies of certain individuals over others but without changing those competencies. 
If screening prevails, then individuals completing job training programs will have higher 
levels of employment and earnings — but their employment will come at the expense of 
other individuals who fail to get these jobs, and employment and productivity in the 
aggregate will not increase. Job training programs generally assume a human capital model, 

19 This is likely to happen if the demand for labor is relatively price-inelastic, in which case any shift in 
the supply function for labor will increase employment only slightly and reduce wages considerably. 



and there is virtually no reference in the evaluation literature to the possibility that signaling 
might explain any positive outcomes. 

• Because experimental evaluations are expensive and cannot rely on normal census 
methods of data collection, they have typically been evaluated for only a relatively short 
period of time after participants complete them — for example, 30 months in the case of the 
recent JTPA evaluations (Bloom et al., 1994), 3 years in the case of the GAIN program 
(Riccio, Friedlander, and Freedman, 1994), 4 years in the case of the experimental 
JOBSTART program (Cave et al., 1993). The longest job training evaluation collected 
information from several welfare-to-work programs 5 and 6 years after the completion of 
the program (Friedlander and Burtless, 1995). These evaluations are therefore quite 
different from the usual analyses of eatings patterns by levels of education, which 
typically analyze individuals between the ages of 18 and 65 and can genc/ate age-earnings 
profiles describing the effects of education on individuals at different ages. 

The period of time is critical because of the question of whether any potential 
benefits increase over time or degrade. In the pattern typical of age-earnings profiles for 
different education levels, for example, a level of schooling may not generate any real 
increase in earnings for several years, during which an individual is searching for an 
appropriate job; then the benefits tend to increase over time, peaking somewhere during the 
period between age 45 and 55 before declining as retirements begin. Similarly, in job 
training programs one might expect a decrease in earnings during the program itself, as 
individuals are forced to leave any employment they might have; then, perhaps following a 
period of job search when earnings art itill Jow, one would hope that earnings compared to 
those of the control group would be higher, and perhaps would continue increasing as the 
greater skills from the job training program enable individuals to advance in their jobs 
compared to the control group. However, a different possibility is that short-term job 
training programs push individuals into low-quality employment without improving their 
skills, so that there are short-term employment benefits that disappear after a short period 
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— leaving experimentals no better off than controls in the long run, and potentially even 
worse off because off the period of low earnings during the program itself. (This may be 
especially dangerous with job search assistance, which is designed to help individuals find 
jobs but without improving their skills.) The difference between these two possible patterns 
can be detected only with information about earnings several years after a program ends — 
and unfortunately many evaluations have not lasted long enough to collect such 
information. The available results on effects over time are reviewed in Section IIL6 below. 

• The predominance of experimental approaches has overshadowed other methods 
of understanding job training programs, particularly the use of qualitative and ethnographic 
evaluations that might provide better insights into why programs succeed or fail. The earlier 
evaluation literature of CETA and the welfare experiments of the 1970s include some 
qualitative studies, in which researchers would observe programs carefully, interview 
participants at length, and otherwise try to determine what life in a program was like for its 
participants. The purpose was not only to get a better sense of what programs are like — 
the "texture of daily life", or the "lived experience" of programs, as ethnographers might 
say — but also to develop better information about how programs are implemented, what 
precisely goes on in them, and why they might be ineffective. In recent examples of 
qualitative ethnographies and case studies, for example, Hull (1994) has described the 
amount of teaching about on-the-job relationships (in addition to technical skills) that 
occurs in a banking program; Kalman and Losey (forthcoming) have analyzed how a 
workplace literacy program fails to live up to its self-conception as an innovative, worker- 
centered program; Gowen (1994) has described the turmoil in a workplace literacy 
program; Grubb and Kalman (1994) have described how the dominant teaching methods in 
work-related remedial programs undermine their effectiveness; and investigation:, based on 
interviews have suggested that certain behavioral problems make job-keeping (rather than 
job-finding) a problem among the chronically unemployed (Quint, Musick, and Ladner, 
1994). This last study is particularly interesting because it examined the lives of 50 women 
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enrolled in New Chance, which was also evaluated with random assignment methods (see 
Section IH.3 and Table 13). They found that those enrolled in the program were 
enthusiastic ?.bout it; but their progress into employment was slow and uneven partly 
because of the problems caused by living in highly disorganized families and communities. 

One rationale for qualitative studies, then, is that they can provide explanations for 
the outcomes determined by quantitative analyses. Many of the reasons I offer in Section 
IV for the small benefits of job training programs are based not on formal results from 
random-assignment experiments but on less formal case studies and observations of job 
training programs. Formal quantitative evaluations are necessary, then, because only these 
methods can demonstrate the effects on employment and earnings of job training programs; 
but qualitative studies are necessary too, to understand why some programs work and 
others don't and to clarify how existing programs might be improved. Unfortunately, these 
two traditions of research are not well-integrated: the qualitative examinations typically 
collect no information about effects on earnings and employment, and the quantitative 
analyses rarely carry out qualitative studies. 

The final drawback of random-assignment evaluation, of course, is that it is 
expensive. It can therefore be applied to large-scale evaluations of national programs of 
considerable policy importance — but it cannot be applied routinely, and cannot be applied 
to small programs, to many experimental efforts, or to local programs deciding what mix of 
services or which specific providers they should use. This means that job training 
programs have typically been subjected to two quite different kinds of "evaluation": 
random-assignment evaluations of great sophistication and cost, performed largely for 
federal policy-makers deciding how to establish federal guidelines and legislation; and 
locally-collected information about effectiveness, like the performance measures required 
by JTPA and information about caseloads collected in local welfare programs. This kind of 
local information, which is much cruder and susceptible to local manipulation, is used to 
monitor local programs, to impose sanctions on local programs that are out of compliance 
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with performance requirements, and in some cases to make local decisions about 
effectiveness. In the only effort to calibrate these local evaluations with random-assignment 
evaluations — to see, for example, whether local programs with strong results on 
performance measures also have strong results from random-assignment evaluations — 
there proved to be no correlation between the two (Doolittle et al., 1993, p. 10). This 
suggests that performance measures are virtually useless for making rational decisions 
about effectiveness — even though they provide political protection because they make 
JTPA seem like a performance-driven program. 

There is little question that the quality of evaluations has increased substantially 
over the past twenty years. Job training programs — and particularly those associated with 
the welfare system — have been the subjects of what is probably the most sophisticated 
policy-oriented analysis in the United States. However, given the complexity of social 
programs and the variety of job training programs, in a country as large and diverse as the 
United States, it should not be surprising that these evaluations have failed to answer all the 
important questions about job training programs. Indeed, given the variety of programs and 
the variation among localities in how they are administered, it is amazing that the existing 
evaluations come to so much agreement about the effects of different programs — the 
subject of the next section. 
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III. THE EFFECTS OF JOB TRAINING PROGRAMS: 
RECENT FINDINGS 



Given the volume of evaluations that have been performed over the past twenty 
years, it is difficult to summarize them in ways that convey both the findings in a literal 
sense and the practical importa^ A the findings. In many cases, job training programs 
have had statistically significant effects, or benefits that outweigh costs, and therefore 
appear to be worth doing — but the effects have been so small that they can have little real 
influence on the course of people's lives, or on the continuing need for welfare programs, 
or on the future development of those enrolled in youth programs. In order to convey the 
findings, I will present a number of tables replicating some of the most important and most 
recent findings, interpreting these as well as summarizing earlier results that are of less 
interest because of the lower quality of the evaluations and because they describe older 
programs no longer in existence. 20 

In Section ELI I present results for CETA and JTPA programs, the major job 
training programs. In Section m.2, I present results for wel fare-to- work programs, while 
Section m.3 reviews several experimental programs. Section m.4 then describes what is 
known for different groups within the population, while Section m.5 does the same for 
different types of services. Section m.6 describes the results over time, to ascertain 
whether any benefits of job training programs persist or decay. Finally, Section m.7 
presents some findings about variations among specific providers, because of findings of 
how much variation there is among different providers in their effectiveness. 

The results in these sectionspresent information about the effects of job training 
programs, regardless of cost. In addition, some (but by no means all) of these evaluations 
have carried out cost-benefit analyses, comparing the effects on earnings with costs of 

AU As mentioned above this is not a formal meta-evaluation; for a meta-evaluation, see Fischer (1995). See 
also the recent review of education and job training programs in What's Working (and What's Not) (Office 
of the Chief Economist, 1995). 
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programs to see if they are "worth doing" not in the sense of increasing the employment 
and earnings of participants, but in the sense of generating benefits that outweigh costs. 
These cost-benefit analyses are surveyed in Section £Q.8. 

1. Mainstream Job Training Programs: 
The Effects of CETA and JTPA 



A wide variety of job training program preceded CETA, because of the variety of 
funding sources and the great latitude local programs had. Partly for this reason, it is 
difficult to compare evaluations of different programs, because the services they offered 
and the intensity (or duration) or programs varied so much. In addition, the methodology 
of evaluation varied as well. The earliest evaluations tended to use conventional regression 
methods using quasi-experimental control and experimental groups: that is, a regression 
describing earnings (or the log of earnings) as a function of individual characteristics plus a 
variable describing program participation (or several variables, if data were available on the 
^ intensity of the program) would be estimated. Table 2 presents a summary of the early 

evaluations of job training programs, distinguishing the results for classroom training, on- 
the-job training, the Job Corps (a residential, year-long program for youth that is much 
more intensive than any other job training program), and adult basic education (ABE), a 
form of remedial reading and writing instruction. The results generally suggest positive 
£ effects, generally higher for females than for males, with a rough average of effects in the 

order of $250 to $300 per year. Given inflation between the early 1970s and the present, 
such benefits would be worth about $900 to $1,000 per year in 1994 dollars. However, 
because of unmeasured selection effects and problems with regression to the mean, 
mentioned above, such regression methods cannot possible control for the true differences 
| between those enrolled in programs and others not-enrolled, so these estimates must be 
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considered over-estimates of the true effects. 
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When manpower training programs were consolidated in the CETA program in 
1973, the kinds of services offered and their administration became somewhat more 
standardized. In addition, the evaluation of these programs expanded substantially, through 
two different avenues: the Youth Knowledge Development Project, which generated a large 
number of qualitative studies of CETA programs; and the generation of the Continuous 
Longitudinal Manpower Survey (CLMS), which followed a random sample of CETA 
enrollees from 1975 on. Individuals in the CLMS survey were then matched to comparable 
individuals from another data set, the Current Population Survey, using different matching 
methods; then, as in earlier evaluations, regression methods were used to disentangle the 
effects of personal characteristics (e.g., gender, race, years of formal education, age, prior 
labor market experience, and the like) from the effects of program participation. 

Table 3 presents the results of several studies using the CLMS data. The benefits of 
participation were generally higher for women; indeed, several studies found statistically 
insignificant effects for men. In general, the conclusion has been that CETA piograms 
increased earnings for women, from about $500 to $1,000, though there is too much 
uncertainty to know whether there were increases for men. For youth, the effects were 
generally zero or even negative. In addition, a smattering of evidence suggests that 
classroom training and on-the-job training were more effective than work experience and 
public service employment, where individuals were employed in public service jobs at 
minimal wages; for example, Taggart (1981, p. 282) concluded that classroom training 
increased earnings in 1976 by $350 or 10%, on-the-job trainees gained $850 or 18 percent 
(declining to $600 the second year), and those in public service employment gained $250 
the first year and $350 the second; individuals in work experience programs actually lost 
earnings. However, the various studies are too contradictory to be very sure about this 
result. 

The different studies, varying in their analyses of the same basic data set, were 
most remarkable for the range of findings: for adult women, the different studies showed 
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variation from no earnings gain at all to $1,300, while for men estimates ranged from 
negative $700 to $691 . One of the most serious problems, aside from the ubiquitous 
selection effects, is nicely illustrated in Figure 1, from one of these CETA evaluations 
(Bloom and McLaughlin, 1982). The year before enrollment, CETA clients show a 
pronounced dip in earnings, compared to the control group; for men, job training simply 
restores them to the level of earnings of the control group, though for women there is a 
slight increase above these earnings of the control group. If the earnings dip is merely a 
transitory component — for example, caused by bad luck, or a temporary spell of 
unemployment — then it is clear that job training programs do nothing for men, though 
they have modest effects for women (about $800 - $1,300 in this particular study, higher 
than in most others). If, however, the earnings dip was something connected to a more 
permanent reduction in earnings capacity, then the benefits of CETA would be considered 
higher. Disagreement about which of these is true in partly responsible for the range of 
estimates in Table 3. One conventional conclusion from the range of estimates, therefore, 
was that quasi-experimental methods were not powerful enough to detect the possible 
benefits of job training programs, and that experimental methods were advisable (Barnow, 
1986). But in part this conclusion resulted from the facts that if there had been any benefits, 
they must have been modest since very large benefits would have been detected even with 
quasi-experimental methods and with substantial variation among different studies. 

A special program is the Job Corps, a particularly intensive program for youth. The 
Job Corps is predominantly a residential program in which youth live in a center away from 
home, and receive a variety of academic instruction, job training, and various other social 
services for an entire year. It has always been the most expensive job training program, 
costing about $15,000 in 1994 dollars, and has represented the most serious kind of 
intervention for those youth judged in the greatest need. As the results in Table 4 indicate, 
based on a quasi-experimental evaluation using a matched comparison group, it has had 
positive effects on employment rates and on overall earnings, though not on wage rates, 



and it has also served to reduce crime among those enrolled. (The findings that wage rates 
have not increased suggest that the Job Corps did not increase the productivity of its 
members; but increased employment led to higher earnings, indicating that the Job Corps, 
like many other job training programs, affected persistence in the labor force instead.) 
Furthermore, benefit-cost analyses have shown that, despite the high costs of Job Corps, 
the values of these benefits outweigh the costs (Table 20). These results, which represented 
some of the earliest benefit-cost analyses of job training, provided some hope that even 
intensive job training programs, for individuals with the greatest barriers to employment, 
would be worth doing. 

In 1983 CETA — which had come under fire as an ineffective and politically- 
manipulated program — was replaced by JTPA. The two major changes intended to 
increase effectiveness were the development of performance measures which local 
programs would be required to meet, and the requirements that local administering boards 
(call Private Industry Councils, or PICs) have at least 50 percent of their members from 
private business, an effort to make job training responsive to the needs of employers. In 
addition, public service employment — which had been criticized as "make work" — was 
eliminated from the services provided. 

With the passage of CETA to JTPA, the evaluations based on CLMS ceased. In 
their place, a random-assignment evaluation of JTPA was planned. The results of this 
study, which are widely regarded as definitive because of the design and complexity of the 
evaluation, 21 describe employment effects for individuals from 16 specific programs across 
the country, both 18 and 30 months after leaving the program. 

Table 5 presents the most basic results from this evaluation. As in earlier findings, 
the impact is higher for adult women ($1,176, or a 9.6 percent increase in earnings) than 



11 There is still a problem of external validity despite the sophistication of the experimental designs: the 16 
programs were volunteers, and though they did not differ in any obvious way from JTPA programs as a 
whole it is still possible that they are more effective than randomly-selected programs would be. 
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for men (who experienced a $978 or 5.3 percent increase in earnings). 22 However, while 
these results are statistically significant, and the benefits of JTPA proved to outweigh their 
costs for both adult men and women (see Table 22 below), in another sense the benefits are 
small: for women, who might have to support a family, the program increased earnings 
only trom $12,241 to $13,417 over these 30 months, an average annual gain of $470. 
Even for those who enrolled in the program, the increase was only $735 per year — not 
enough to move individuals out of poverty, for example, or enable them to leave welfare. 
To be sure, the long-run effects might be more positive, as I explore in Section IH6 below; 
but clearly JTPA did not provide a substantial boost to the earnings of either women or 
men. 

Furthermore, the effects of these programs on youth are zero or even negative. The 
negative findings come about because the youth enrolled in the program largely withdrew 
from employment during the period of training, and the lower earnings during the period of 
enrollment were not offset by increases in earnings after completing the program. (Since 
labor market experience is one of the most important criteria for hiring in the sub- 
baccalaureate labor market — more important even than many educational credentials below 
the baccalaureate level [Grubb, Dickinson, Giordano, and Kaplan, 1992] — the long-run 
effects for youth enrolled in ineffective training programs may be even worse,) The 
negative findings are particularly discouraging for the worst-off youth — those who had 
been arrested prior to enrolling in the program. To be sure, these results for youth reflect 
only a subset of youth in JTPA, and are therefore not necessarily comprehensive. 23 
Nonetheless, these are very discouraging findings which confirm in many ways the results 



22 These results indicate the problems even with an experiment: Because many of those individuals 
assigned to enroll in the program failed to show up, the numbers of cnrollees actually undergoing job 
training was smaller than the number assigned. The benefits for those enrolling were evidently higher than 
those assigned, as Table 5 shows. 

2 ^ These JTPA evaluations examined only the Title II-A programs, not the Title II-C programs that require 
local SDAs to spend 40 percent of their funding on special youth programs. 
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for CETA in Table 3, and they have caused many analysts and policy-makers to call for 
eliminating JTPA programs for youth. 24 

Other effects of JTPA can be seen from Tables 6 and 7. It clearly increased the 
proportion of individuals with a GED or high school diploma, though by only trivial 
amounts for young males; it seemed to reduce the receipt of welfare benefits for women 
and female youths (though these effects are insignificant), though it increased welfare 
benefits among adult males; and it reduced the arrest rate for young males who had not 
been arrested prior to enrolling in the program. But these effects are quite modest and 
uncertain; and in the case of increasing the rate at which individuals earn a GED, there is 
substantial uncertainty about whether this credential improves subsequent employment. 25 
Thus examining benefits other than those related to employment doesn't improve the 
conclusions much: there are benefits, but they are modest for some groups, missing for 
others, and in still other cases — e.g., the increase in welfare benefits to males, as well as 
the overall negative effects for youth — they operate in the wrong direction. 

Overall, the results of the JTPA evaluations are sobering. They reveal modest gains 
for adult men and women — in the order of $500 to $750 for women per year, and perhaps 
$400 to $650 for men 26 — but earnings increases that are essentially zero or even negative 
for youth. The results are for adults are statistically significant and, as I will point out in 
Section m.8 and Table 22 below, JTPA programs are also "worth it" in the sense that their 
benefits outweigh their costs. But the benefits are not significant in any practical sense — 
they are too small to change the life conditions of those who have enrolled in job training, 

24 In addition, these results do not cover the summer youth programs, which provide young people short- 
term employment during the summer. There is little evaluation of these programs, though generally 
subsidized work experience has not been effective in improving employment of low-income youth (Office 
of the Chief Economist, 1995, p. 11). 

2 5 The General Education Diploma (GED) is a multiple choice test that is intended to be the equivalent of a 
togh school diploma. However, the evidence about its effects is mixed: a careful analysis by (Cameron and 
Heckman, 1993) found that it had no effect on employment once individual differences were controlled 
though a recent re-analysis of these data by Murnane Willett, and Bordett (1994) found more positive but 
still small effects. Other evidence (e.g., Quinn and Haberman, 1986) suggests that the GED does not 
increase the rate at which individuals enroll in postsecondary education. 

26 These are substantially smaller than the gains recorded for CFTA, as might be expected given the upward 
bias of the quasi-experimental results summarized in Table 3. 
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to enable many of them to leave the welfare rolls, or to escape poverty. This kind of 
finding, replicated in many other studies, leads to the puzzle addressed in Section IV: why. 
after about 25 years of developing job training programs, are the benefits of job training so 
small? 

2. Welfare-to-Work Programs 



As I mentioned in the introduction, the idea of providing services — including 
education and training — to enable welfare recipients to move into employment and off 
welfare extends back at least to 1962. In addition to the voluntary work programs 
established during the 1960s, the federal government under Richard Nixon allowed states 
to experiment with their own welfare-to-work programs. Although the evaluations of these 
programs were primitive, to say the least, they generated an enormous amount of rhetoric 
Q on behalf of welfare-to-work programs. For example, while he was governor of 

California, Ronald Reagan established a community work experience program (CWEP) in 
£ which welfare recipients were required to work in community service jobs in amounts 

related to their grants (i.e., they were required to "work otf ' their welfare grants). The 
California CWEP program was a complete failure: it was able to enroll only a tiny fraction 
(0.2%) of welfare recipients, and it failed to meet every single one of its employment 
objectives (Employment Development Department, 1976). Nonetheless, Reagan cited the 
program as a success virtually every time he discussed welfare, and he used the presumed 
"success" of this program to press for an expansion of welfare-to-work programs. 

While the early welfare-to-work experiments tended to emphasize work, rather than 
education and job training, the emphasis shifted somewhat in the 1980s. During the Reagan 
administration, states were allowed to implement a series of experiments in their welfare 
| programs, incorporating a mix of work requirements and services (including education and 

training) in order to reduce their welfare populations. These experiments varied widely, to 
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be sure, though most of them ended up emphasizing job search rather than either job 
training or mandatory work: the political and practical difficulties involved in forcing 
welfare recipients into employment — when they have children as well as deficient skills 
and, in many cases, other personal characteristics that made employment difficult — was 
too great for mandatory work to ever be very substantial. 

As was true of state programs in general, the state welfare-to work programs of the 
1980s varied substantially in the services they provided. For example, of the five state 
programs described in Table 8, the Arkansas and San Diego programs provided job search 
workshops, followed by work experience in public and private agencies; Virginia 's 
program began with a period of job search followed by either work experience, education, 
or training; Baltimore included a variety of different services including education, training, 
job search, on-the-job training, and work experience; and West Virginia required 
community work experience, potentially of unlimited duration. As was true of other state 
programs, then, these five tended to emphasize job search assistance and work experience 
programs over education or job training; a few states did allow education and job training, 
especially as initiated by the welfare recipient. Thus the services in these welfare-to-work 
programs overlap those provided by JTPA; but the evaluations of these experiments should 
not be considered evaluations of training itself. 

Several of the state welfare-to-work programs were evaluated with random- 
assignment methods; the results were revealing in their own right, and they also set the 
stage for more widespread programs enacted in 1988. Table 8 summarizes the results from 
5 state experiments. Four of the five increased the amount of employment; two increased 
earnings by statistically significant amounts (by $560 per year in San Diego and $156 in 
Arkansas), with two more increasing earnings by amounts that were not quite statistically 
significant ($176 in Baltimore and $108 in Virginia). As a result, in three of the five states 
the total amount of welfare payments fell, by amounts per year ranging from $84 in 
Virginia to $192 in California. But the effects of these programs in moving welfare 
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recipients off welfare was negligible: in none of these five states was the likelihood of 
being on welfare reduced. 

These evaluations also allowed a number of other useful conclusions, in addition to 
dispelling the notion that "nothing works". The least effective program was that of West 
Virginia — but this was generally attributed to the weak state economy in a largely rural 
state with few employment prospects. This result suggested that welfare-to-work programs 
(and job training programs more generally) could not be expected to have much effect in 
weak economics, perhaps including rural economies generally. Second, these experimental 
programs appeared to provide the greatest help to the least well-off or the least job-ready 
individuals — for women compared to men, and for individuals without prior employment 
compared to those with a previous work history (Gueron, 1987). When added to similar 
results from CETA, these results suggested that job training programs ought to concentrate 
their efforts on individuals with the most barriers to employment — the very opposite of 
"creaming". 

In these welfare-to-work experiments, therefore, there were benefits of some kind 
in virtually all states, and — as Section m.8 and Table 21 clarify — the benefits exceeded 
the costs of operating these programs. In addition, the benefits varied among states, and the 
San Diego experience suggested that earnings increases could be substantial — at least the 
order of magnitude of benefits recorded by JTPA. However, the effects we still modest 
by almost any standard: earnings increases were quite small, ranging between $100 and 
$200 per year in most states; the reductions in welfare payments by states were similarly 
small; and these programs had no effect whatsoever in reducing the number of families on 
the welfare rolls, the goal that is at the heart of welfare-to-work programs. The results 
could be read as either supporting or undermining the continuation of welfare-to-work 
programs, but they indicated that such efforts made very little difference to the lives of 
welfare recipients and provided only trivial savings for taxpayers. 
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One other evaluation of a state welfare-to-work programs was carried out using a 
non-experimental design (Nightingale et al., 1991). The Massachusetts Employment and 
Training (ET) Chojces program was evaluated comparing ET participants to a control group 
of other welfare recipients who did not participate but who were matched on the basis of 
race or ethnicity, age, sex, age of youngest child, region of the state, and whether the 
individual was in a one- or two-parent family; then regression methods were used to 
control for variations among individuals (and between experimental and the comparison 
groups). The results, summarized in Table 9, are of the same order of magnitude as the 
experimental results in Table 8 but are somewhat more positive: the increase in the 
probability of employment was greater than for the five states in Table 8, the average 
annual earnings increase ($780) was larger, and the reduction in annual welfare payments 
($307) was larger than the largest reduction among the five states (where California had a 
reduction of $192 per year). However, it is difficult to know whether these differences are 
due to random variation among states — the possibility that Massachusetts was simply at 
the upper end of a range of outcomes — or to the greater variety of services offered in ET 
Choices (which allowed welfare recipients to enroll in job training, postsecondary 
education, and remedial education in addition to job search assistance and supported work 
experience), or to the non-experimental design of the evaluation, which probably resulted 
in outcomes that were upwardly biased. However, the results confirm once again that 
welfare-to-work programs can affect both earnings and welfare grants — though the effects 
are modest, and unable either to improve the lives of welfare recipients substantially or to 
end the need for welfare itself. 

In 1988, in part based on a favorable reading of the experimental program 
evaluations, the early welfare-to-work programs were expanded in the Job Opportunities 
and Basic Skills (JOBS) program. JOBS required states to fund welfare-to-work programs 
(partly with federal matching funds) with a mix of work requirements, job search 
assistance, work experience, training, education, counseling, child care, and other 
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supportive services. Again, states set up a variety of programs, varying in their job 
requirements and their mix of services including job training, education, and less intensive 
services like job search assistance. However, many states implemented their JOBS 
programs through JTPA as a matter of state policy, while in other states local JOBS 
programs accomplished the same thing by convening a variety of local providers and 
emphasizing the use of the existing JTPA system 27 — in effect commingling JOBS with 
the dominant job training system. 

So far, two evaluations of JOBS program have been completed: one of the 
California program called GAIN (Greater Avenues for Independence), and another of the 
Florida program, Project Independence. 28 The GAIN evaluation, using random- 
assignment methods, investigated the effects of programs in 6 counties (out of 58), varying 
by their urban/rural characteristics, the state of the local labor market, and the nature of 
local programs. Of the many services GAIN provided, the increase caused by enrolling in 
the program was most substantial for job search activities (since 28.5 percent of the 
experimental group received these services, compared to 3.9 percent of control group 
members) and remedial education (defined as adult basic education or GED training, 
received by 29 percent of experimental but 5.4 percent of controls) — but vocational 
training and post-secondary education did not increased markedly as a result of GAIN. 
This confirms the bias within welfare-to-work programs: even though welfare recipients 
are allowed to participate in self-initiated education and job training programs lasting up to 
two years, in practice the emphasis on short-term job search assistance and on remedial 
education 29 has reduced the use of lengthier education and training programs. 



11 See Lurie and Hagen (1994), summarizing a large number of monographs on the implementation of 
JOBS; Grubb et al. (1990); and Grubb and McDonnell (1991). 

A random-assignment evaluation of JOBS programs in 6 states is also underway, but the results on 
outcomes will not be completed for at least another year. 

29 The emphasis on remedial education comes from the fact that individuals in GAIN go through a precise 
sequence of stages. At an early stage their basic reading and math are evaluated, and those who are deficient 
are directed to remedial education. In general, one of the discouraging (but unsurprising) findings of welfare- 
to-work programs is how many welfare recipients lack basic academic skills. 
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The overall results are summarized in Table 10, differentiated into effects for single 
parents (almost all women) and heads of 2-parent families (almost all men). Over the three- 
year period of the evaluation (from early 1988 to mid- 1990) the program increased the 
probability of employment for both groups; increased earnings for both groups, by $471 
per year and $370 respectively; and reduced the amount of welfare payments, by $305 per 
year and $389 respectively. GAIN also reduced the likelihood of being on welfare by 3 
percentage points among the single parents, since 55.5 percent of the controls but only 
52.5 percent of the experimental group were on welfare at the end of the third year; 30 
however, there was no effect in moving heads of households off welfare. To be sure, these 
are average effects across all six counties; the substantial variation among counties 
(reviewed in Section m.7) indicated that some counties were more effective than others. 
Overall, however, the results are quite consistent with the pre- JOBS results: welfare-to- 
work programs relying on the most modest services — in this case, job search assistance 
and remediation — can increase employment and earnings and result in some welfare 
savings, but the effects are small and the reduction in the numbers of families on welfare is 
trivial. 

The results from Project Independence are different in several respects. This 
welfare-to-work program was really a job search program, since it provide relatively little 
education or training. Like many other programs, it did increase the amount of employment 
(as Table 1 1 shows), and also increased earnings — though the average increase of $1 14 
per year was very small. It also succeeded in reducing the rate at which individuals received 
welfare, and in fact the savings in welfare payments were greater than the increases in 
earnings — a finding that proves to affect the cost-benefit results (in Table 24 below) 
substantially. However, two other findings were of greater interest. The evaluation found 
substantially higher benefits to mothers with children over the age of 6 — who have many 

JU This figure nicely illustrates the amount of movement on and off welfare that occurs "naturally", 
without any special programs: almost half of the single parents in the control group left welfare by the end 
of the third year. 
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fewer problems related to child care, scheduling, sick days, and the like — than to mothers 
with children age 3 to 5, as the results in Table 1 1 clarify. This finding reinforces the 
notion that certain individuals — tliose with substantial barriers to employment — may 
not benefit from job training, an idea that I will examine further in Section IH.4. 

In addition, the evaluation revealed a common vulnerability of job training and 
welfare programs. During the evaluation, a worsening recession in Florida caused welfare 
caseloads to increase while funding stayed the same; the result was that the resources 
available to clients who enrolled late in the evaluation — the "late group" — were 
substantially smaller. Consistent with this, the increases in earnings and the declines in 
welfare payments were significant only for the "early group", not for the "late group" — 
clarifying that the fiscal conditions of welfare-to-work programs may influence the results 
substantially. The implication is that individuals who point to certain exemplary welfare-to- 
work programs as evidence of what job training might accomplish — for example, the 
Riverside program described in Section m.7 and Table 19 below — neglect the more likely 
possibility that fiscal constraints will lead to low-quality programs with no effects. 

One hint about how this program affected behavior came from an examination of 
attitudes and values. Those who enrolled in the program were more likely to agree that 
"even a low-paying job is better than being on welfare", were less likely to think than 
mothers should stay home with their children rather than working, and had lower 
reservation wages. Even though many of these differences were statistically insignificant, 
they all support the notion that welfare-to-work programs can change attitudes, replacing an 
acceptance of welfare with a greater commitment to work — consistent with one of the 
intentions behind welfare-to-work programs. But Project Independence also reduced the 
overall income of those enrolled by a small amount, and it decreased the fraction of those 
saying that they were satisfied or very satisfied with their overall standard of living, from 
45.7 percent among the control group to 41 .9 percent among those in the program. Project 
Independence therefore represents a relatively conservative approach to welfare-to-work 
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programs, in which the costs of welfare to taxpayers declines (see also Table 24) at the 
expense of welfare recipients themselves. 

3. The Effects of Experimental Programs 

There has been a common pattern in the United States of experimenting with social 
programs, trying out promising practices on a small scale before expanding them to more 
universal programs. Indeed, the history of job training efforts can be interpreted as part of 
this larger history. The development in the 1960s of manpower training programs located 
outside the schools was in part an effort to develop novel efforts that bypassed the 
presumed deficiencies of the educational system; these "experiments", with the roughly 
positive results summarized in Table 2, were then institutionalized in CETA and then 
JTPA. Similarly, the current round of welfare-to-work programs in the JOBS programs 
emerged from welfare-to-work experiments operated by states during the 1980s, which 
were in turn based on fledgling efforts of the 1960s and 1970s. 

There have been several other, more self-consciously experimental programs 
developed with private and foundation funding that have contributed to our knowledge of 
"what works". There have, not surprisingly, been a very large number of experimental 
programs, each with its partisans — partly because private foundations in the United States 
often operate by developing an experimental approach (or discovering an experimental 
approach underway in some corner of the country) and then replicating it elsewhere, 
sometimes with some kind of evaluation. Some of these efforts have proved not to be 
especially effective because they cannot be replicated — for example, some are dependent 
on the high energy and charisma of a founding leader, and do not work once they are 
operated by other people. However, others have been designed to be replicated, and have 
been intended as tests of particularly approaches to employment and training. In contrast to 
the mainstream job training and welfare-to-work programs — which have provided a 
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variety of services, often without much thought or guidance about what might be 
appropriate — these experimental programs have developed clear models of what 
employment-related services ought to be provided to specific population groups, and have 
then worked to make sure that these models are carefully implemented. One might expect, 
therefore, that these experimental programs would be of higher quality, and would generate 
more favorable outcomes, than do the mainstream programs reviewed in the previous two 
sections. 

In this section I present the evidence for 4 programs: two for young mothers with 
children, the Minority Female Single Parent Demonstration (MFSP) and New Chance, a 
program for young poor mothers; and two for youth, JOBSTART for high school dropouts 
and the Summer Training and Employment Program (STEP) for youth at risk of dropping 
out of high school. 

The Minority Female Single Parent Demonstration (MFSP): This 
experiment was an effort to devise an effective employment program for a group that has 
suffered special difficulties in gaining access to stable employment, and that constitutes a 
group likely to like on welfare for long periods of time: minority women with children but 
without husbands. The program, which took place in four sites around the country, 
emphasized the provision of remedial education, on the assumption that basic academic 
skill deficiencies were a barrier to employment, and also provided an extensive array of 
services like child care, counseling, guidance in managing daily problems, and help in 
finding jobs after the program. Two programs (the Atlanta Urban League, AUL, and the 
Opportunities Industrialization Center in Rhode Island, OIC) emphasized classroom 
courses in academic skills along with job-specific training; one (the Center for Employment 
Training in San Jose, California, or CET) supposedly emphasized very specific job skill 
training; and Wider Opportunities for Women in Baltimore (WOW) emphasized general 
employability skills (including basic academic competencies as well as communications 
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skills, motivation, and discipline) through classroom activities (Gordon and Burghart, 
1991; Rangarajan, Burghardt, and Gordon, 1992). The demonstration, funded partly by 
foundation funds, was evaluated with random assignment methods. 

Because the four MFSP sites were somewhat different in their services, the 
outcomes were reported for each of the four programs. Table 12 presents the most 
important results from the four. The obvious story is that three of the four programs had no 
influence whatsoever on employment and earnings, or on the receipt of welfare benefits, 
over the period of the evaluation. However, one program — the Center for Employment 
Training — had substantial effects, increasing the amount of employment by 13 percent, 
increasing wage rates by 1 1 percent, and increasing average earnings by 25 percent or $101 
per month ($1,212 on an annual basis) — a huge effect compared to those of other JTPA 
programs in Table 5, for example, or the GAIN results in Table 10. Like GAIN and earlier 
welfare-to-work programs, MFSP did not decrease welfare payments substantially, nor did 
it decrease the likelihood of being on welfare. 

The MFSP Demonstration results can be read either positively or negatively. On the 
negative side, three of four programs that were funded to create exemplary job training 
programs, with relatively lavish funding and special attention to the design of the 
programs, failed to have any effects at all. On the positive side, the substantial effects of 
CET — confirmed in the JOBSTART evaluation reviewed below — suggested that well- 
designed programs can work. Indeed, the success of the CET program in this evaluation 
has been the subject of a kind of publicity campaign to trumpet the success of a particular 
approach. In the interpretation of the evaluators and the founder, the success of CET was 
due to its efforts to integrate remedial education and vocational skill training, and they then 
began to promote a model of job training that depends on such an integration (Burghardt 
and Gordon, 1990; see also Literacy and the Marketplace, 1989). This may in fact be a 
worthwhile approach, as I argue in the Conclusion of this monograph; but the success of 
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CET is due to many factors in addition to the provision of both remediation and job skill 
training, as I point out in Section 111.7. 

Overall, however, the results of the MFSP Demonstration are remarkably consistent 
with other evaluations, especially those of GAIN: job training programs on the average 
have modest positive effects on employment and earnings, 31 very little effect if any on 
welfare payments, and no effect on the likelihood of being on welfare — though individual 
programs (like CET) may have much more substantial effects. 

New Chance: The New Chance program was an experiment conducted between 
1989 and 1992 at 16 sites in 10 states. Similar to the MFSP Demonstration, it concentrated 
on young mothers, almost all of them high school dropouts. The New Chance model 
assumes that poor young mothers require a range of supportive services in addition to 
remedial education and job training, because of having a variety of personal and 
psychological barriers to employment. The different sites provided remedial education, 
preparation for the GED, career exploration, instruction in job-finding skills, health 
education, family planning, and life skills; in a second phase, more employment-focused 
services were provided including job training, work experience, and job placement 
assistance. Child care was also provided, as well as some services to children including 
heath care. Services were more intensive than the usual JTPA or JOBS services, scheduled 
for 20 to 30 hours per week up to 18 months. 

Table 13 presents the findings of random-assignment evaluations 18 months after 
entering the program. (Subsequent results will examine results after 42 months; if it takes 
time for positive results to emerge — as seems to be true for certain programs, as reviewed 
in Section EQ.6 below — they will not be apparent in the findings available so far.) New 
Chance was successful in increasing attendance in GED programs and college, and in 
increasing the proportion of mothers who earned a GED and some credits toward and 

31 Across all four programs in Table 3, the average effect on earnings was 6 percent or about $17 per 
month ($208 per year) 
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postsecondary credential. Surprisingly, however, the program did not increase scores on a 
test of basic skills (the TABE, or Test of Adult Basic Education), and it left 72 percent of 
those enrolling (compared to 70.2 percent of the controls ) reading at a ninth grade level or 
below — suggesting that it is possible to earn a GED without improving academic 
competencies. 32 For a wide range of other outcomes, however, New Chance made no 
difference in the short run. Indeed, if anything it appeared to have negative consequences: it 
appeared to increase pregnancies, though these were balanced by increased abortions so 
that the number of births stayed about the same 33 ; it reduced employment, though not 
significantly so; it reduced earnings significantly; and it increased welfare slightly. To be 
sure, it is possible that these results reflect withdrawal from employment during the period 
of the program itself, and that the early declines in employment and earnings will be 
reversed after young women get into more stable employment in their third and fourth years 
after enrolling (as for the JOBSTART program summarized below). However, the early 
findings, roughly consistent with the MSFP Demonstration results, are not at all 
encouraging since the only real benefit has been an increase in a credential — the GED — 
which has little effect on either employment or subsequent education. The results are 
particularly discouraging given the high cost of the programs of about $9,000 per person. 

JOBSTART: The JOBSTART program was an experimental effort to create 
effective programs for youth age 17 to 21 who have dropped out of high school. While 
high school dropouts have been considered a problem in the United States virtually through 
this century, recently the problem of dropping out has been interpreted as even more 
serious because of evidence that their employment and earnings have been falling relative to 
those of high school completers (Levy and Murnane, 1992; Grubb and Wilson, 1992), and 

JZ Others have suggested that it is possible to pass the GED with an eighth or nineth grade reading level 

perhaps an indication why earning a GED has so little influence in either employment or in enrolling in 
postsecondary education. 

In the American context where there is vociferous opposition to abortion, it would be politically 
detrimental to any program to increase both pregnancies and abortions. 
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because dropouts constitute a majority of the population that is permanently poor or on 
welfare — as distinct from those who are poor or use welfare only temporarily. 
Conventional job training programs for youth have not been successful, as the results for 
JTPA in Table 5 illustrate quite starkly — though these dismal results were not available 
when JOBSTART was initiated in 1985. 

The JOBSTART demonstration > )ok place in 13 sites around the country, using 
JTPA funds.; it was modeled roughly on the successful Job Corps program (reviewed in 
Table 4 above), though it was less intensive and non-residential. Each site provided 
remedial education, vocational skill training, job placement assistance, and various support 
services like child care, transportation, counseling, and instruction in work readiness and 
job skills; sites were required to offer at least 200 hours of basic education and 500 hours 
of job training, making them more intensive than conventional JTPA programs. Three 
different models of service provision were followed: concurrent programs provided 
remedial education and occupational training concurrently; sequential/in-house programs 
provided remedial education and then vocational skill training; and sequential/brokered 
programs provided remedial education and then referred participants to other programs for 
vocational skill training. 

The JOBSTART Demonstration was evaluated using random-assignment methods 
over a four-year period. The most important results are summarized in Table 14. One major 
effect is that the program increased the rate at which drop-outs received a GED — a result 
that is not surprising because most of the programs emphasized the GED. However, given 
the limited effects of the GED in increasing employment or access to postsecondary 
education, this impact may not be of much value. Indeed, over the four-year period the 
effects on employment and earnings were insignificant, both for the total sample and for 
selected sub-groups. As in other evaluations, the proportion of women receiving welfare 
did not decrease overall — indeed, it increased for women with children — and the 
amounts of welfare did not decrease. Nor did rates of pregnancy or giving birth fall 
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significantly — a special concern because of the negative effect childbearing has on poverty 
and welfare dependency; indeed, for mothers entering JOBSTART, rates of pregnancy and 
giving birth increased during the program. One positive result is that rates of arrest 
appeared to fall, as did drug use (significantly so for hard drugs excluding marijuana). 

The results in Table 14 are dismal, and suggest that well-designed job training 
programs of moderate cost do not work for youth at all. However, these average effects do 
mask some potentially positive findings for the longer run. A common pattern was for 
employment and earnings to fall in the first year of the program, while individuals were 
enrolled in education and job training, with employment increasing in the second year while 
earnings and earnings increased in the third and fourth years — suggesting (from the fourth 
panel of Table 14) z pattern in which those enrolled in the programs increase their earnings 
about $400 per year over the long run. This pattern emerged for most sub-groups including 
men, mothers, and other women, and was particularly marked for men arrested before 
enrolling in JOBSTART — for whom earnings increases were $1,129 in year 3 and 
$ 1 ,872 (and statistically significant) in year 4 — and for youth who left school for 
academic reasons, for whom earnings increases were $726 in year 3 (and statistically 
significant) and $592 in year 4. In addition, the finding that drug use decreased as a result 
of the programs suggests that more positive effects might show up in the long run as some 
individuals avoid drug-related arrests and drug-motivated unemployment and create stable 
employment records instead. 

The other positive finding is that one of the 13 sites — the Center for Employment 
Training in San Jose, the successful site in the MFSP Demonstration — had statistically 
significant increases in earnings: Over the four-year period, the experimental group earned 
$32,959 compared to $26,244, an increase of 25.6 percent and $1,679 per year (and 
$3,044 per year over years 3 and 4). Unfortunately, disaggregating by sites makes the 
findings more dismal, if anything. Seven of the 13 sites had negative outcomes — about 
what one would expect by chance alone — and two of them had very large negative effects 
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1 in the range of $6,200 over the four years. Eliminating the one clear success of CET as a 

| special case, the other 12 sites averaged negative effects of $1 ,393 over four years and 

$21 1 over years 3 and 4, making it difficult to conclude that the long-run effects could be 
| positive. The CET program may be a success story, for the special reasons I analyze in 

Section D0L7, but otherwise it is difficult to find much hope for youth programs in the 
| JOBSTART results. 

i 

STEP: The Summer Youth and Training Program (STEP) is an interesting effort to 
£ devise a coherent program for youth in their junior and senior years of high school — that 

is, those who have not yet dropped out (as JOBSTART youth had) — but who were 
£ considered because of their poor academic performance to be likely to drop out or, for 

girls, to become pregnant. The design of STEP was intended to combat the problem of 
"summer effects" — the fact that the academic achievement of many youth (especially low- 
£ income youth) regresses during the summer. Therefore the program provided two summers 

of activities; in each, students took remedial classes for half the day and. worked for the 
£ remainder of the day, providing an opportunity to learn other work-related skills and to 

earn money. The inclusion of both school-based and work-based components in theory 
enabled students to see the connections between the two, since they could have seen how 
| academic competencies are learned on the job and could have used the work component 

because of the concern with pregnancy as a barrier to school completion, a half-day each 
£ week was devoted to issues about responsible sexual behavior, contraception, and other 

aspects of personal responsibility. In addition, some support services (counseling and 
guidance, and some tutoring) were available during the intervening school year. Because it 
combined both school-based instruction and work, the STEP design incorporated several 
elements of the school-to-work model that has emerged recently in the United States, and 
| that I describe in Section V. In addition, the STEP sponsors took great care in the 
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replication of the program in about 85 sites, to make sure that the important elements of the 
model were in place everywhere. 

The STEP program was evaluated with a random-assignment approach in which 
students enrolled in both the school and the work component were compared to others 
enrolled in the work component only; thus the design tests the additional effects of the 
summer schooling component. The effects on reading and math scores and on knowledge 
of contraception after the first summer were positive; gains during the second summer were 
also positive though somewhat smaller. However, 3 1/2 years after enrolling in the 
program, STEP youth experienced the same dropout rates, rates of postsecondary 
enrollment, employment rates, and rates of teenage pregnancy as did the control group 
(Walker and Vilella-Velez, 1992; Grossman and Sipe, 1992). One widely-cited conclusion 
from the STEP experiment is that, while it is possible to improve academic performance 
through a short-term program, long-term results and effects on more fundamental behavior 
like employment and pregnancy cannot be changed with a short-term intervention that 
leaves the rest of schooling and the general environment of poor youth unchanged. 

Overall, the results of these experimental programs are disheartening. Although two 
of them identified a particular program — CET in San Jose — as having particularly Strang 
effects, the programs on the average had no effects or even negative effects. To be sure, the 
populations included in these experiments were among the most difficult to employ, since 
two focused on young mothers with children and two others on low-income youth. But the 
results clarify that the modest effects achieved in the general JTPA and JOBS programs 
cannot be improved upon merely by paying somewhat closer attention to the design of the 
program and to their implementation. 
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4. The Effects of Job Training for Specific Population Groups 



The results in the previous three sections concentrate on the overall effects of job 
training programs. However, an important question is whether the effects vary for different 
population groups — for example, if they are greater for men versus women, or for adults 
versus youth, or for whites compared to black or Hispanic individuals, or for those who 
are the most employment-ready compared to those who have multiple barriers to 
employment. If it were possible to find certain groups for whom job training programs 
work better than for others, it would be possible to target resources to those groups and 
increase the overall effectiveness of the limited resources available for job training, or to 
provide different kinds of services to different groups. 

In the early evaluations of manpower projects (e.g., Table 2), there were no clear 
conclusions about which groups benefit the most from job training. However, the CETA 
evaluations found greater effects for women compared to men, and for adults compared to 
youth (e.g., in Table 3, elsewhere in Barnow, 1986, and Bloom and McLaughlin, 1982). 
In fcddition, the benefits appeared to be higher for those with little labor market experience, 
compared to those with substantial experience before enrolling in a program (Bloom and 
McLaughlin, 1982) — suggesting that job training programs might benefit those with the 
least skills and experience, while those with more skills and experience would simply cycle 
in and out of employment on their own and not be helped by job training programs. 34 

The results of the welfare-to-work experiments in the 1980s tended to confirm the 
findings that the lest job-ready individuals would benefit the most. Employment increases 
were generally greater for women on welfare, for example, and for individuals without 
prior employment histories (Gueron, 1987, p. 28). A more sophisticated reading of the 
evidence suggested a kind of tripartite result, presented in Table 15: individuals within 

34 This conclusion is also supported in Fischer's (1995) meta-analysis of job training programs. However, 
he provides no numbers to back up this claim, and the most recent evaluations — the GAIN study by 
Riccio, Freidlandcr, and Freedman (1994), and the JTPA study at 30 months by Orr et al. (1994), both of 
which weaken his results — have not been included in his analysis. 
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these welfare-to-work programs who were the most job-ready — who were first-time 
welfare recipients — generally did not benefit (those in Tier 1); and those individuals with 
the most serious barriers to employment — those on welfare more than two years and those 
with no prior earnings — had low and often insignificant benefits (in Tier 3). A group in 
the middle, with some prior time on welfare and with low (not non-zero) earnings (in Tier 
2) appeared to gain the most (Friedlander, 1988; Gueron and Pauly, 1991, Ch. 4). Such a 
conclusion is consistent with earlier findings that the most-job-ready individuals do not 
benefit much from job training, but also with the negative results of the experimental 
programs reviewed in Section m.3 that focus on the most disadvantaged. This kind of 
finding also supports a kind of "triage" policy, in which the most job-ready would be 
denied access to job training programs (in contrast to the practice of "creaming"); the most 
disadvantaged, with the greatest barriers to employment, would also be denied access to 
conventional programs or — because this practice might be difficult to implement because 
of political and moral considerations — would enroll in more intensive and expensive 
programs than are typically offered. Conventional programs would then concentrate on the 
group in the middle, for which the most substantial gains at lowest cost might be expected. 

The more recent JOBS evaluations have also generated results for specific groups 
within the population. For the California GAIN program, one set of results tended to 
confirm the kind of tripartite finding from the earlier experiments: recipients of welfare who 
were moderately disadvantaged earned more, and received lower welfare benefits, than did 
either more disadvantaged recipients or new applicants (Riccio, Friedlander, and Freidman, 
1994, Table 4.6). Unfortunately, for other groups this conceptually straightforward 
conclusion became less clear, because the effects on sub-groups varied so much among the 
six counties. For examples, individuals were classified according to whether they needed 
basic education or not, since those in need of basic education may need remediation before 
they can benefit from job training, job search assistance, or any other services — and 
therefore may be more expensive to return to employment and less effective in finding 



52 



employment after any job training program. Overall, both groups benefited from GAIN, in 
both increased earnings and reduced welfare benefits. However, one of the six counties 
(Riverside) increased the earnings of both groups; two (Alameda and San Diego) increase 
the earnings of those not needing basic education, but not the other group; and two (Butte 
and Tulare) increased earnings of those needing basic education but not those with better 
academic skills. 

In addition, long-term welfare recipients benefited in three or four of the six 
counties, contrary to the results in Table 15; but new applicants also benefited in two of the 
four counties that collected such information. The programs in Riverside and San Diego 
benefited both those with and those without prior employment; on the other hand, Alameda 
and Los Angeles benefited only those without prior employment, while Butte and Tulare 
benefited only those with prior employment. Thus these results tend to cast doubt on the 
"triage" solution suggested by earlier evaluations: at least under some conditions, in some 
programs, the most disadvantaged individuals can benefit from welfare-to-work programs, 
while in other cases the principle beneficiaries are the least disadvantaged. 

The evaluation of the Florida program, Project Independence, also muddied the 
waters somewhat. As Table 1 1 shows, mothers with children age 3 - 5 benefited much less 
than did mothers whose children were 6 and over — consistent with the practice of 
"creaming" that selects only the individuals with the fewest barriers to employment; and 
those considered job-ready (in terms of their education and prior labor market experience) 
benefited somewhat more than those with less education and experience. However, 
individuals who had been on welfare for two years or more — who were presumably less 
job ready — benefited much more than did those on welfare less than two years (or those 
who were first-time applicants). Thus the effects for specific groups provided little clear 
guidance for selecting individuals, save for the demonstration that mothers with young 
children benefit less than others. 
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The results of evaluating JTPA using more sophisticated random assignment 
methods have confirmed the greater impact for women compared to men (table 5) — 
though the differences were not as great as for the earlier CETA evaluation (e.g., Table 3) 
— and the finding of zero or even negative effects for youth confirmed the earlier 
conclusion that job training might be effective for adults but not for youth. (The findings of 
greater effects for women are consistent with the earlier findings of greater effects for the 
least job-ready — since women are likely to have less labor force experience, and to have 
children that complicate their employment — but the larger effects for adults rather than 
youth contradict this pattern.) For other sub-groups, the results were generally statistically 
insignificant, and therefore the most cautious conclusion is that there were no differences 
among different groups in the benefits of job training (Orr et al, 1994, Ch. 5, especially 
Exhibits 5.8, 5.9, 5.19, 5.20). However, the more serious problem is that the differences 
which can be detected did not fall into any real pattern. Some of the differences confirm the 
conclusion that the least job-ready individuals, or those with the greatest barriers to 
employment, benefited the most: 

• Women receiving welfare benefited more than those not on welfare, and those on 
welfare for two years or more benefited more than those on welfare for a shorter period of 
time. 

• Women who worked less than 13 weeks in the year prior to the program benefited 
more than women with less work experience. 

However, other results suggested than the most job ready individuals benefited the 

most: 

• Hispanic women, more likely to have low levels of education and to be non- 
English speaking, benefited less than white or black women. 

• Women with a high school diploma or GED benefited more than those without 
such a credential. 
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• Women who earned $4/hour or more in their last job benefited more than those 
with lower wages or no employment, as did those with family income over $6,000 per year 
compared to those with lower income levels. 

• Women with a spouse present benefited significantly, out those without a spouse 
present and with a child under 4 — precisely the population like to have been included in 
the MFSP Demonstration and New Chance, one should note — also benefited. 

• Men receiving no welfare benefited more than those on welfare. 

• Men who earned $4/hour or more in their last job benefited more than those with 
lower wages or no employment, as did those with family income over $6,000 per year 
compared to those with lower income levels. 

• Men with a wife present — who are presumably more responsible or more stable 
— benefited more than those without a spouse. 35 

For youth, differences among sub-groups are even more difficult to detect because 
most of the results were zero (or statistically insignificant) or negative; again there is no 
clear pattern to any differences that exist. Overall, therefore, it is difficult to conclude much 
from these results about the impact of job training on different sub-groups, and they 
certainly undermine the earlier notion that job training is most beneficial for those with the 
greatest barriers to employment. 

After a great deal of research, therefore, the effectiveness of job training programs 
for different sub-groups remain murky. Overall, women benefit more than men, and adults 
more than youth, for whom the results have often been zero or negative. But at least in 
some programs, the least job-ready individuals benefit, while in others the opposite is true. 
This in turn suggests that any particular strategy for selecting applicants — for example, the 
efforts to "cream" or select the most job-ready, or the contrary efforts to target job training 
on the most disadvantaged individuals, or the "triage" solution suggested by Table 15 — 
cannot be generalized. 



35 This is the only one of these comparisons that was statistically significant. 
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5. The Effects of Specific Services 



Given the variety of services provided by job training programs, another salient 
question is whether certain services are more effective than others. Typically, services vary 
not only in the nature of what is provided but also in their cost and intensity: classroom 
training (in either remedial academic skills or in vocational skills) is usually of longer 
duration, and costs more, than does job search assistance, which does not pretend to 
enhance skills; and the duration and content of work experience and on-the-job training can 
vary substantially. The intensity of programs offered under various job training programs 
has varied substantially, from very short programs of half-day sessions lasting 10 weeks or 
so, or barely more than 30 hours of instruction, to the year-long residential Job Corps 
program with its wide variety of services, costing in current dollars about $15,000 per 
person. If certain services are ineffective, or if certain expensive services are less effective 
than other more expensive ones (or the other way around), it is important to know that in 
order to devise the most effective programs — or, alternatively, the most cost-effective 
programs. 

The early evaluations (e.g., in Table 3) were often interpreted as supporting the 
greater effectiveness of on-the-job training over classroom training (e.g., Taggart, 1981, p. 
282). Some reviews of the CETA experiences similarly concluded that on-the-job training 
was more effective than classroom training, which in turn was more effective than work 
experience by itself, without a training component or associated classroom instruction 
(Taggart, 1981, pp. 282 - 288). This kind of finding was consistent with the evidence that 
CETA increased earnings not by increasing wage rates — or, in economist's terms, by 
increasing productivity — but by increasing rates of employment, something that might be 
improved more by on-the-job training and its attention to work-related behavior than by 
classroom instruction and its emphasis on cognitive and vocational skills. Similarly, 
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I Barnow (1986), reviewing a greater number of studies, concluded that public service 

g employment and on-the-job training had greater effects than classroom training, and that 

work experience had no effects (or even negative effects). In addition, the early evaluations 
£ of CETA concluded that longer classrooms training programs were on the whole more 

effective than shorter programs, especially for women (Taggart, 1981, p. 103 ff.); in part, 
{ the strong results from the year-long, residential Job Corps program (Tables 4 and 20) 

strengthened the case that longer, more intensive programs would be more effective. 
The JTPA evaluation, using more stringent random-assignment methods, also 
£| examined the effectiveness of different kinds of services provided. Individuals were 

assigned to three different kinds of services: classroom training in occupational skills; on- 
the-job training/job search assistance, where individuals typically were enrolled in job 
» search assistance and then sometimes found on-the-job training or unsubsidized jobs; and a 

* category of "other" services that might include basic or remedial education, job search 

J| assistance, work experience, and miscellaneous other services. For adults, individuals with 

classroom training received an average of 551 hours of services costing $3,174, 
J substantially higher than for either OJT/JSA (averaging 222 hours and $1,427 per person) 

or "other" services (204 hours at $1,1 16 per person). For youth, "other services" 
(averaging 328 hours at $2,016 per person) were almost as intensive as OJT/JSA (3 17 
hours at $2,755), while again classroom instruction was the most intensive approach 
(averaging 596 hours at $3,305). To put these figures in perspective, a one-year certificate 
program in a community college might take 30 weeks of time with about 25 hours per 
week, or 750 hours — so the most intensive of these programs involved only two-thirds of 
the hours of the least intensive vocational program, and the most common services were 
only one-third the intensity. 

However, the effects of these different kinds of services are not related to the 
£ intensity of services. 36 As the results in Table 16 indicate, the benefits are greatest for 
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J,D Because individuals were assigned to services based on which were considered appropriate, the 
characteristics of clients varied among services and these results are not experimental. 
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"other services" for women, followed by OJT/JSA; for men, OJT/JSA is the most effective 
(though the difference is statistically insignificant). These results tend to confirm earlier 
findings about the superiority of on-the-job training to classroom instruction. For youth, 
however, the results are inconclusive because of the lack of statistical significance, though 
classroom training appears more beneficial (or, in reality, less harmful) than other services. 

In the early evaluations of welfare-to-work programs, it was not possible in any 
formal sense to evaluate the effectiveness of different services because individuals were 
assigned to various services by non-random methods. The states varied in the mix of 
services they provided, leading to the conclusion that different kinds of programs could all 
achieve the kinds of modest results found in Table 8 (Gueron, 1987, p. 30). However, 
because many control group members in these experiments had access to education and job 
training programs through educational institutions and JTPA, the greatest difference in 
services between experimental and control groups were typically in the short, job search 
assistance provided in welfare-to-work programs. 

The design of the GAIN program in California reinforced this kind of conclusion. 
For many participants (those with a high school diploma or a GED, or who passed a 
literacy test), GAIN required job search assistance before any other services; therefore the 
dominant service provided by GAIN was the short-term JSA, and the generally positive 
effects in Table 10 can be attributed to this kind of service rather than any other. However, 
the differences among the 6 different counties evaluated in this study did suggest some 
other conclusions. In four of the six counties, the presence of large earnings increases 
among those in need of basic academic education suggest that remediation is an important 
component of job training; on the other hand, the absence of earnings gains for these 
needing in remediation in two counties that emphasized remediation (Alameda and San 
Diego) indicates that basic skills instruction cannot guarantee success. Other results in 
Alameda also suggested that vocational training and postsecondary education could have an 
effect — though two counties (Riverside and San Diego) produced large earnings increases 
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without increasing vocational training or postsecondary education. The findings therefore 
suggest — as Taggart concluded from the CETA results earlier — that a variety of different 
service strategies can be used in effective programs, without strong evidence that one 
particular service is preferred over another. 

On the other hand, the results of the Project Independence program (Table 1 1) 
indicate the importance of at least some minimal level of services to outcomes. The group 
that enrolled late in Project Independence, after a recession forced increases in caseloads 
without a corresponding increase in resources, did not benefit at all — presumably because 
they received very little from the program. (The cost per person for individuals in the "late" 
cohort was slightly less than $900, less than almost any other job training program.) Even 
at its best Project Independence was not a particularly intensive program, since it provided 
little more that job search assistance to most clients; but even so the results suggest that 
results cannot be expected when resources dip below some critical level. 

The experimental programs described in Section m.3 all involved much more 
intensive and carefully-devised programs, incorporating a range of services. The 
disappointing outcomes therefore give little support to the idea, associated also with the Job 
Corps, that a range of services and more intensive programs are necessary to overcome the 
multiple handicaps to employment of the least job-ready groups. On the other hand, the 
evidence for the Center for Employment Training in San Jose, reviewed in Section m.7, 
seems to support the conception of comprehensive services. This idea remains an attractive 
one, and I return to it in the Conclusion — but neither the national JTPA evaluation nor the 
GAIN study nor the experimental programs of the past few years provide conclusive 
support for this idea. 

One possible interpretation is that either of two very different strategies may work 
in job training programs. The first involves relatively inexpensive approaches. On-the-job 
training has proved to be effective in CETA and JTPA; similarly, job-search assistance 
supported in many welfare-to-work programs appears to be modestly effective, even 
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though it is relatively inexpensive and of short duration. Both services are designed to get 
individuals into employment quickly and to socialize them to the norms and values of 
employment, though neither do anything to enhance the basic cognitive or vocational skills 
of individuals who lack such competencies. The fact that job training programs have 
proved to be effective more because of enhancing employment rates, rather than increasing 
wage rates (and presumably productivity), suggests that the effectiveness of such services 
is related to getting individuals into employment quickly. Whether such limited services 
have much effect over the long run, or whether they merely substitute one group of under- 
prepared workers for others, are questions that the existing evaluations cannot answer. The 
second effective strategy has been associated with more intensive services, including the 
kinds of remedial education supported by GAIN and various experimental programs, and 
often combined with vocational skills training and supportive services. This intensive 
approach is designed to increase the competencies of individuals who are enrolled in job 
training programs, rather than to push them into low-wage jobs, and to increase their 
productivity and wage rates over the long run. These two strategies serve somewhat 
different purposes, to be sure. More to the point, they can be linked — as I argue in the 
Conclusion. 



6. The Patterns of Benefits Over Time: Expansion versus Decay 



The majority of job training programs are short term, lasting perhaps 15 to 20 
weeks; and they are generally self-contained, so that individuals enrolling in them do not 
typically enroll in other programs to continue the process of education and job training. 37 
Therefore job training programs are usually "one shot" efforts to get individuals into 



This may happen informally; in particular, some individuals claim that locating job training programs 
in community colleges provides individuals easier access to the educational programs of these institutions; 
see, e.g., Grubb et al. (1991). However, my point is that job training programs are not structured to lead to 
subsequent training and education opportunities. For a proposal to link education and training programs in 
"ladders", see also the Conclusion. 
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employment, rather than the beginning of a longer period of education and job training (as, 
for example, postsecondary education can be). This is turn raises the question of what the 
long-term benefits of short-term programs are. If, consistent with the human capital model, 
job training programs provide their clients with real skills (cognitive, behavioral, or 
vocational) that increase their productivity and are valued in the labor market, then one 
would expect to see a permanent increase in wage rates and earnings. If in addition the 
initial enhancement of skills allows individuals to enter "careers" with subsequent on-the- 
job training that further increases productivity, one would expect to see wage rates and 
earnings continuing to increase over time — as happens in the age-earnings profiles 
associated with different levels of formal schooling. If, however, job training programs 
merely push individuals into the labor force without increasing their skills substantially, 
and fail to gain them access to "careers" or other positions with long-run possibilities for 
advancement, then the effects of such programs may be short-term, and even with positive 
short-term benefits the long-run benefits may be essentially zero. Indeed, various education 
programs have been found to be subject to this kind of "fade-out" or decay of benefits — 
including the STEP program described in Section BDL3 above — from which many have 
concluded that sustained interventions are necessary to improve the life chances of low- 
income students. 

Thus it is important to examine the benefits of job training programs over time, to 
see whether any initial benefits are sustained (or even increase) or whether they decay. 
Unfortunately, this has not been easy because most evaluations last a comparatively short 
period of time, and the long-term effects are therefore unknown. However, a few studies 
shed some light on this problem. 38 In Table 17 for example, the pattern of earnings for the 
JTPA evaluation, summarized in Table 5 above, is displayed for different periods over the 
30 months of the evaluation. During the first six months, earnings increases are essentially 



38 In addition to the experimental results reported here, Gcraci (1984) used the CLMS to measure the 
relation between longer-term earnings and short-term indicators. In general the correlations arc quite low, 
indicating the inaccuracy of short-term measures of outcomes. 
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zero, because this is the period when individuals leave the labor force to enroll in the 
program. In the next 12 months (months 7 to 18), the earnings advantage increases to an 
average of $68 per months among women and $45 among men; in the next twelve months 
this advantage is sustained at about $71 per month among women and men (and becomes 
statistically significant among men). There is no evidence in these figures of further 
expansion of benefits, but neither do they indicate any "fade-out". (For youth, not 
surprisingly given the negative results in Table 5, there is no pattern of benefits at any time 
during the 30 months.) Similarly, the results for JOBSTART in Table 14 suggest that 
earnings of those enrolling are initially lower (as are hours worked), but then increase to a 
plateau of about $400 per year more than those in the control group — though employment 
rates, after increasing in year 2, seem to fade away. However, the Project Independence 
results in Table 1 1 indicate some "fade-out" or decay in benefits, since increases in 
earnings and employment, and decreases in welfare payments, are greater in year 1 than in 
year 2. 

The evaluations based on the longest period of time involved four of the welfare-to- 
work experiments begun during the 1980s, and included programs in Virginia, Arkansas, 
Baltimore, and San Diego (Friedlander and Burtless, 1995). Figure 2 presents the effects 
of these programs on annual earnings over the five-year period, as well as on welfare 
payments; the actual figures for employment rates and earnings are given in Table 18. The 
dominant pattern, in the top panel of Figure 2, is for earnings increases to be trivial in the 
first year, to increase substantially in year 2, to remain substantial in years 3 and 4, but then 
to fade in years 4 and 5. Similarly, welfare savings are typically trivial in year 1, increase, 
but then fade in years 4 and 5. (It's important to note that the period of greatest effect 
includes years 2 and 3, which is precisely the period covered by months 19 to 30 in the 
JTPA results in Table 17.) In the more detailed information in Table 18, there appears to be 
"fade-out" in the employment rate for all programs except perhaps in Arkansas, and "fade- 
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out" in earnings except perhaps in Baltimore. 39 Over a period of time, therefore, these 
results indicate that the kinds of welfare-to-work programs instituted during the 1980s, and 
largely continued in the JOBS program, do increase earnings and reduce welfare payments, 
but only in the moderate run; over the long run they leave individuals with employment 
rates and earnings no higher than welfare recipients who have not enrolled in such 
programs, and they do not permanently move individuals off welfare. 40 

There are two possible exceptions to the pattern of decline over time. The only 
welfare-to-work program with stable benefits was the Baltimore program, where earnings 
increased to year 3 and then stayed about the same in year 5 (see also Figure 2). One 
possible explanation is that the Baltimore program stressed human capital development 
through education and training, more than job search intended to accelerate job finding — 
and the development of enhanced competencies increase earnings capacity over the longer 
run (Friedlander and Burtless, 1995, p. 144). In addition, the GET program in San Jose — 
a program that is clearly a special case — showed relatively stable increases in earnings 
over a five-year period, though increases in wage rates and in employment rates tended to 
fall (Zambrowski and Gordon, 1993). 

In general, however, these results do not suggest that expansion of benefits over 
time occurs as a result of job training program — as happens, typically, when individuals 
complete educational credentials like Associate degrees and baccalaureate degrees. In the 
short run like the first year, benefits of job training programs are typically non-existent 
because individuals withdraw from the labor force — and short-run evaluations (like that of 



Jy There are technical problems in each of the two figures that suggest the lack of fade-out: the year 6 
employment rates for Arkansas are actually extrapolated from two quarters of data, and the year five earnings 
in Baltimore are extrapolated from three quarters of data. These two crucial numbers could therefore be 
considered less certain than the figures based on four quarters of data — even though the method of 
calculating standard errors, which relies on variation across individuals rather than over time, does not 
consider this. 

40 These results are corroborated by Fischer's (1995) meta-analysis, which concluded that across all studies 
the mean effect sizes for the proportion employed increase gradually until quarter 9, at the beginning of year 
3, but then decay rapidly over the next three quarters. Similarly, the proportion on AFDC declines slightly 
until quarter 6, remains substantial in quarters 7 to 10, and then declines to zero; See Tables A.l and A.2. 
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New Chance in Table 13) are therefore suspect. Over the moderate run, in years 2 and 3 
(and perhaps year 4 as well) the benefits increase; but after that they decay. 

To be sure, these findings can be interpreted either negatively, because of the lack 
of long-run effects of job training programs, or positively (Friedlander and Burtless, 
1995). In the four programs described in Table 18, two of them saved money for 
governments since the reduction in welfare payments over five years ($735 in Arkansas 
and $1,930 in San Diego) outweighed the costs per person (of $1 18 and $920 
respectively). In all four of the programs the increases in earnings over five years were 
larger than costs, so that from a social standpoint the benefits to all individuals 
— recipients plus taxpayers — outweighed the costs. In this sense these welfare-to-work 
programs were "worth doing", even though they did not reduce the welfare rolls and did 
not prepare individuals to leave poverty in any permanent sense. 

7. The Variation in the Effectiveness of Different Programs 

One of the clear implications of many evaluations is the unsurprising finding that 
there is substantial variation among specific programs. The important question, however, is 
whether these differences can be explained, either by the characteristics and quality of the 
programs themselves, by the characteristics of individuals accepted into these programs 
(which is particularly important if some programs engage in "creaming"), 41 or by the local 
economic and employment conditions of particular programs. Otherwise, one would 
normally expect some random variation in the outcomes of programs — that is, purely by 
chance some would have better outcomes than others — but such variation would not be 
useful to program administrators or policy-makers trying to improve the quality of job 
training programs. 



Of course, in random-assignment evaluations the effects of creaming will be eliminated because the 
experimental and the control groups have roughly the same characteristics by construction. 
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An example of variation among specific programs than may be purely random 
comes from the JTPA evaluation (the evaluation summarized in Table 5 above). At the end 
of 30 months there were substantial differences among the 16 sites examined: the increases 
in earnings varied from $2,628 to negative $2,033 among women, and from $5,310 to 
-$2,637 among men. Among youth, for whom the average effects were negative, the 
variation was even more marked since one program achieved statistically significant 
earnings increases of $3,372 for females and a whopping $9,473 among males who had 
not been arrested (Orr et al., 1994, Exhibit 4.5 and 4.16). However, partly because of 
small sample sizes in the 16 sites, these differences across sites were not statistically 
significant; and an analysis that tried to attribute the differences across sites to program 
conditions, economic conditions, and personal characteristics yielded no statistically 
significant results. 42 

The evaluations of welfare-to-work programs during the 1980s found benefits 
smaller in West Virginia than in other states (see Table 8), a difference generally attributed 
to poorer economic conditions in that state. In the GAIN evaluation, one finding was that 
the Riverside program was consistently more effective than any of the other five counties 
(Table 19); indeed, the increase in earnings of nearly 50 percent for individuals in Riverside 
county is one of the largest effects ever found for any job training program, and the 
correspondingly large net benefits in Riverside suggested that well-designed programs 
could save taxpayers money by reducing welfare costs sufficiently (see Table 23 below). 
Conversely, however, the GAIN results indicate that — even within a state that imposes a 
certain uniformity on its welfare-to-work programs — there can be localities (like Los 
Angeles and Tulare, the latter a rural county) where programs may have no effects on 
earnings at all. Indeed, the variations among sites were even greater than those revealed in 
T^ble 19: within counties, there were substantial differences between the results in 
outlying, largely suburban offices, where the benefits were substantial, and those in inner- 

42 Bloom ct al. (1993, Exhibit 7.12). This analysis was based on the 18-month results, which may be too 
early, and was not replicated for the 30-month results. 
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city offices, where the benefits were either insignificant or even negative — a result 
suggesting that the demographic composition of those enrolling might be responsible for 
the differences. However, a regression-based analysis controlling for such demographic 
characteristics did not eliminate the substantial differences among counties and among 
offices within counties (Riccio, Friedlander, and Freedman, 1994, Table 8.2); and in any 
case some counties were successful with those not needing basic skills while others 
succeeded with those who were less job ready, suggesting that different counties could be 
successful with different groups of clients. 

Nor did differences in local economic conditions — measured, for example, by 
unemployment rates and the growth rate of employment — explain differences; in 
particular, the effects of the Riverside program were remarkably consistent even though 
economic conditions in that county varied substantially during the period of the study, and 
varied in addition among office. Nor was there any obvious explanation of differences 
among counties based on the kinds of services offered. In the end, the evaluators 
concluded that the success of Riverside was due to the combination of practices there: a 
strong message to participants about the importance of getting into jobs early; a strong 
commitment to job search and job placement efforts; a mix of job search, education, and 
training; and a commitment to enforcing mandatory participation of all eligible welfare 
recipients (Riccio, Friedlander, and Freedman, 1994, Ch. 8). Another observer has 
concluded that the high expectations of the program staff were responsible (Bardach, 
1993), and still e thers have attributed the success to the energy and charisma of the 
director. 

The evaluation of Project Independence also found substantial differences among 
counties, ranging from earnings increases over two years of $1 ,333 to earnings losses of 
$570 (Kemple, Friedlander, and Fellerath, 1995, Table 6. 13). However, these differences 
were not statistically significant — that is, it is possible that they were generated simply by 
chance — and they were not related in any obvious way to variation in labor market 
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conditions, services available, or patterns in program participation. Such results add to the 
conclusion that, while there are differences among programs that appear substantial, it is 
not yet possible to explain these differences by labor market conditions or by program 
characteristics that could be affected by policy. 

The experimental programs reviewed in Section III.3 above also found substantial 
differences among specific programs. One results in particular was striking: two 
independent evaluations, one of JOBSTART and one of the Minority Female Single Parent 
Demonstration (in Table 12), found that the Center for Employment Training in San Jose, 
California, was much more effective than the other programs. The MFSP evaluators 
concluded that the success of CET was due to its practice of linking (or "integrating") both 
job-specific skills and remedial education, along with its attention to job placement and its 
availability of child care at the site (Burghardt and Gordon, 1990). 

However, the truth is probably much more complex and — like Riverside — the 
success of CET probably lies in a combination of factors. 43 First, the CET program has 
been in San Jose for a long time, and has long-standing connections with employers that 
facilitate its finding placements for its students. 44 Second, the program concentrates on 
Hispanics, and most of the instructors are both Hispanic and bilingual; the program is 
therefore providing bilingual education in addition to job skills instruction, as well as 
mentoring and acculturation to American practices for those individuals who have just 
immigrated. Third, the site at San Jose performs real work — for example, it operates a 
child care center, a copying business, a cafeteria for the CET members themselves, an auto 
repair shop, and the like, each associated with one of the job training programs — so that 
students are getting work-based training and experience as well as classroom instruction in 
both job skills and remedial subjects. Fourth, the presence of social services at the site — 

43 These comments are based not on the published descriptions of the CET program, but on several visits 
by Judy Kalman and Norton Grubb to CET programs in San Jose and Oakland in the course of research for 
Grubb et al. (1992). 

44 However, it is quite possible that CET has become a trusted supplier of labor to the low-wage 
employers, who then hire through CET rather than other sources; in this case the employment effects of 
CET might be offset by displacement, or reductions in employment from other sources. 
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child care, but also assistance with immigration issues and job placement — is clearly 
important. While CET does provide both remediation and job skills training, and is 
particularly conscious of the need for English language instruction, the two are not 
integrated in any important sense — they take place at different times of the day, and there 
is rarely any mention in one component of the program of the lessons from the other 
component — so the emphasis on CET as an "integrated" program seems misplaced. 45 But 
there are clearly many other positive elements to CET and a broad range of job-related 
services provided, and it is easy to understand why it is so much more effective than other 
job training programs. 

The conclusion from both the Riverside program and CET is that a combination of 
practices distinguishes particularly successful programs. I shall return to this finding in the 
conclusion, since certain current recommendations for job training programs build on the 
findings presented in this section. 

8. Cost-Benefit Analyses of Job Training 

The results presented so far describe the outcomes of job training programs, and 
whether they improve the employment rates and earnings (or other potential outcomes like 
arrest rates, welfare receipt, and fertility behavior). A different way of asking whether job 
training programs are "worth it" is to compare the outcomes to the costs, though benefit- 
cost analysis. Since the late 1970s, many job training programs have been subjected to 
cost-benefit analyses, using generally-accepted methods tc establish values for the net 
present value of the outcomes that could be attributed to the program — that is the 
difference in outcomes between those enrolling in the program and a control group. In most 
cases the additional earnings of those enrolled and the reduction in welfare benefits 

45 The pattern of teaching remedial education and job skills in different and independent segments is typical 
of job training programs that claim to "integrate" the two; they really mean that they provide both, not that 
the two are integrated. Like most job training programs, the teaching methods used at CET are quite 
conventional, teacher-directed methods following the practices of "skills and drills". 



represent the major benefits, though in some cases the value of crime prevented and drug 
and alcohol abuse avoided are counted among the benefits (e.g., in the cost-benefit analysis 
of the Job Corps, in Table 20). To be sure, there are numerous intangible benefits of job 
training, especially that associated with greater economic independence and reduced use of 
welfare programs; and there are probably uncounted costs as well, particularly the 
opportunity costs of mothers with young children, who might otherwise be caring for their 
children. 

Typically these analyses calculate costs and benefits separately for different groups 
of potential beneficiaries, usually including those enrolled and the "rest of society" (or 
taxpayers), in order to capture the distributional effects of job training. That is, it is 
possible that such programs benefit those enrolled while taxpayers lose, or conversely that 
the benefits to taxpayers through decreased welfare and crime costs outweigh the costs of 
the programs, even though those enrolled do not earn enough more to offset the loss of 
welfare benefits. 46 The hope of job training programs, of course, is that both taxpayers and 
those enrolled will enjoy benefits in excess of costs. 

One of the earliest and most influential cost-benefit analyses was that of the 
residential Job Corps program, summarized in Table 20. The results of this analysis are 
fairly typical of subsequent analyses: overall the net benefits to those enrolled in the Job 
Corps were positive, because the increased earnings per person ($3,397) were high enough 
to offset the lost of welfare income ($1 ,357), the opportunity costs associated with being 
unable to work during the period of the program ($728), and other costs associated with 
being in the program. But the Job Corps was a net loss to taxpayers, since the very high 
cost of the program ($5,351 in 1977 dollars, and about $15,000 now) was so large that the 
benefits from taxes on increased earnings ($1,255), reductions in welfare benefits and 

46 In these cost-benefit analyses, the treatment of transfer payments is carried out correctly; that is, a 
reduction in welfare benefits is simultaneously a cost to recipients and a benefit to taxpayers, and the overall 
effect of a transfer on society is zero. The only real savings from reducing welfare programs therefore comes 
from reducing administrative expenses, as well as from the intangible benefits of having fewer individuals 
rely on welfare. 
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administrative costs ($1,515) and reduced criminal activity ($2,281) failed to offset it. 
From a social point of view, adding the benefits to Corps members plus taxpayers, the 
benefits outweighed the costs; the general lesson from this analysis was that even an 
expensive job training program could demonstrate its worth in cost-benefit terms. 

Similarly, the welfare-to-work programs developed during the 1980s were 
evaluated using benefit-cost analyses. Table 21 presents the results of a typical and 
particularly influential analysis, the San Diego program, for both a work experience 
program (EWEP) and job search assistance. The results, as for the Job Corps, indicate net 
benefits for those enrolled in the program, as earnings increase by a small amount (e.g., 
$461 for work experience and job search together), but net losses for taxpayers since the 
costs of the program were higher than any reduction in welfare payments and increases in 
taxes from increased earnings. Since from a social perspective the benefits outweigh the 
costs, the results of this experiment were widely cited as confirming that welfare-to-work 
programs were worth undertaking, and were widely cited in legislation for the JOBS 
program. However, the fondest hopes of those hostile to welfare — that such job training 
programs would save money for taxpayers — were clearly not met by the San Diego 
program. 

In the more general analysis of the JTPA program, the cost-benefit results are of 
course consistent with the findings of the outcome studies. For adult women and adult 
men, for whom there were significant increases in earnings, the benefits outweigh the costs 
both for those enrolled in the programs and for society, as presented in Table 22 — though 
again taxpayers lose. But for youths, for whom the effects on earnings were negative 
because of reductions in earnings during the period of program, the program generated net 
losses of participants as well as taxpayers, with losses per person especially high ($2,904) 
in the case of young males. The only conclusion possible is that job training programs are 
a particularly poor investment for youth, and indeed, since these results became public 
there have been numerous proposals to reduce or eliminate job training efforts for youth. 
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The benefit-cost analyses of the GAIN program in California, summarized in Table 
23, confirm that different programs of varying effectiveness can generate different 
conclusions in cost-benefit analyses. This analysis distinguished among three groups of 
potential beneficiaries: welfare recipients enrolled in GAIN; governments supporting the 
costs of GAIN; and taxpayers, who also pay for GAIN through their taxes but who 
experience certain benefits and costs — for example, the output associated with unpaid 
work experience — that do not directly affect government budgets. The results indicate that 
both government budgets and taxpayers lose for all groups of recipients; only in the case of 
recipients not needing basic remedial education — which requires relatively expensive 
classroom training and not simply low-cost job search assistance — did the benefits 
outweigh costs both to recipients themselves and to taxpayers. But the detail for specific 
county programs reveals that the most effective programs — those in Riverside and in San 
Diego — can generate savings for taxpayers as well as substantial net benefits for 
recipients, indeed, in Riverside the benefits to governments and to taxpayers are larger than 
those to recipients. (And, on the contrary, the worst program — that in Los Angeles 
County — actually made recipients worse off, as JTPA did for youth.) From these results, 
therefore, the hopes for welfare-to-work programs lies in their being able to emulate the 
characteristics of the most effective programs since on the average the GAIN program 
generated net losses to society. 

The results for Project Independence are somewhat different than they are for 
virtually any other job training program (see Table 24). Because this program did not 
increase earnings by much, but did reduce welfare benefits, the net effects for those on 
welfare were actually negative. However, because the program did not cost much (an 
average of $1,150 per person) and did reduce welfare costs by a considerable amount 
(about $1,155, according to the figures in Table 24), the program resulted in a small 
savings to taxpayers 47 — quite the opposite of the results for GAIN and most other job 

47 0nc could argue that the net gains to the government budget and to taxpayers in Table 24 arc so small 
that they could easily be negative, given random variation in programs. 
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training programs, where recipients gain but taxpayers lose. However, overall Project 
Independence generated losses to society since the very small gains to taxpayers did not 
outweigh the high losses to welfare clients. 

Of course, any positive cost-benefit results depend on job training programs with 
significantly positive effects on employment. In the case where benefits are essentially 
zero, then of course benefit-cost analysis indicates net losses to society. Table 25 presents 
the benefit-cost results of JOBSTART, which had largely insignificant results (in Table 
14). Consistent with these findings, the net benefits to participants were very small, and the 
losses to taxpayers and to society as a whole were substantial. 

On the whole, however, the results of cost-benefit analyses of job training 
programs are relatively consistent — at least if we ignore the evidence from the least 
effective programs (e.g., JTPA for youth), the most effective programs (like Riverside), 
and the different case of Project Independence. In general, the benefits of job training 
programs outweigh the costs since the modest increases in earnings are larger than the 
modest expenditures per person in these programs. However, these benefits accrue largely 
to those enrolling in job training programs, and governments (or taxpayers) typically do not 
benefit since their costs outweigh any benefits they receive in the form of higher taxes on 
higher earnings, reduced welfare costs, and reduced costs associated with crime and other 
social problems. Overall, then, job training programs are "worth doing" — but it is unclear 
whether taxpayers would continue to support these programs if they realized that they are 
likely to be net losers. 
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IV. The Modest Effects of Job Training: Alternative Explanations 



Overall, the results from nearly thirty years of evaluating job training programs are 
remarkably consistent — surprisingly so, given the variation in the kinds of programs that 
have been supported and the differences in the methods of evaluation. A large number of 
job training programs lead to increased earnings, and the benefits generally outweigh the 
costs — though the increases in earnings are moderate by almost any standards, 
insufficient to lift those enrolled in such programs out of poverty. Welfare-to-work 
programs also increase employment and reduce the amount of welfare, but they rarely 
allow individuals to leave welfare. Furthermore, any benefits probably fade out after four 
or five years: job training programs do not seem to put many individuals on career 
trajectories with continued earnings increases, as formal schooling does. 48 For some 
groups — youth in particular — job training programs generally seem to be ineffective 
(unless perhaps the program is very intensive, as the Job Corps is), and programs appear 
to be more effective for women than for men — but otherwise it is difficult to conclude that 
any particular group benefits more than any other. There are, to be sure, some outliers — 
particularly effective programs like the Center for Employment Training and the Riverside 
GAIN program. But there are spectacular failures as well — including some experimental 
programs with carefully- considered program designs, most job training programs for 
youth, the worst of the GAIN programs, and Project Independence as a whole — that 
actually leave those enrolled worse off and thereby violate the first maxim of intervention: 
"do no harm". 

One can assess the modest outcomes of job training programs in either positive 
terms, as indicating that these programs are worth doing on the average, or in negative 
terms. In my interpretation, however, the results are very discouraging: thirty years of 

48 These conclusions are consistent with Fischer's (1995) meta-analysis. Ke concludes that effects sizes arc 
significantly different from zero, but very small: individuals who receive job training average increased 
earnings of $200 to $540 per year, and a decrease in welfare payments of $200 to $400 — but that these 
effects generally decay over time. 
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experimentation with job training programs have created a substantial number of programs 
whose benefits — for individuals in dire need of employment and economic independence 
— are quite trivial, and are completely inadequate to the task of moving them out of 
poverty, off of welfare, or into stable employment over the long run. The puzzle is why 
such well-intentioned efforts have been so ineffective. In this section I present ten possible 
explanations, based on the results in Section HI, direct observations of job training 
programs, and comparisons with education programs. Such explanations must remain 
speculative, of course, since there is not yet enough evidence about truly effective job 
training program to "prove" what works. Nonetheless, such explanations help provide 
some guidance for recommendations — the subject of the last section. 

1. Small programs and small effects 

The first and most obvious explanation is simply that most job training programs 
are "small": they last a very short period of time, rarely more than twenty weeks; they often 
provide a single kind of service — on-the-job training, or classroom training, or job search 
assistance — rather than a variety of complementary services. Job training administrators 
often take pride in this aspect of their programs: they win say, for example, that they offer 
"Chevrolet" programs compared to the "Cadillac" programs of educational institutions, by 
which they mean that they can get to the same destination at much lower cost; they often 
scorn education programs for being too "academic" and unconcerned with immediate 
employment. 

However, the individuals enrolled in these programs often have multiple problems 
and several barriers to employment: they often lack job-specific skills, general academic 
skills, and the kinds of values (including motivation, punctuality, persistence, the ability to 
work with others) necessary to find and keep employment, and some of them have more 
serious problems like drug and alcohol abuse, physical handicaps, other health problems, 



depression and other mental health problems that may be biological rather than experiential. 
Even when this is not the case, the gap between the needs of those enrolled and the scope 
of programs is sometimes breath-taking; for example, one job training program I observed 
was trying to train Spanish-speaking women to be English-proficient secretaries in a 15- 
week, part-day program. There is, then, a disjuncture between the profound needs of those 
who have not found stable employment and the small size of job training programs — and 
we should not be surprised to find the effects on employment to be trivial. 

It is useful to compare the intensity of job training programs to those of education 
programs, measuring intensity by expenditures. The average JTPA program for adult men 
and women cost about $2,200 in 1987-89, for a period of enrollment of 20 weeks (Bloom 
et al, 1994, Exhibit 2); the cost per person in the GAIN program was about $2,300 in 1993 
dollars (Riccio, Friedlandr . and Freedman, 1994, p. 75). (Of course, many of the 
experimental programs are much more expensive, and the current cost per person in Job 
Corps of about $15,000 is the highest of all.) In contrast, one year of full-time enrollment 
in a community college — approximately 30 weeks — averaged $5,700 (Digest of 
Educational Statistics, 1994, Table 325), which is considerably more than the limited JTPA 
and GAIN programs. On the other hand, a one-year certificate program increased earnings 
by about 15 percent over those of high school graduates, and over a long period of time in 
the labor force (Grubb, 1995, Table 3), rather than increasing earnings only over a four or 
five-year period. Of course, this comparison is not especially fair because the 
characteristics of JTPA and GAIN clients are quite different from those of students 
enrolling in community colleges. These comparisons indicate once again that the typical job 
training program provides less services at substantially lower cost per person, to 
individuals with less education and (often) more personal problems, compared to typical 
postsecondary occupational programs— and it should not be surprising that the effects of 
job training are so small. 
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2. The mistaken strategy of job training 

A second possibility is that the basic strategy of many job training programs, and 
virtually all welfare-to-work programs, is simply the wrong one. Most programs — 
including the successful Riverside program — have stressed moving individuals into 
employment quickly, using job search assistance, work experience placements, and on the 
job "training"; they provide relatively little actual training, despite their name. The 
underlying assumption is that the basic problem of the unemployed is one of job-finding, 
and that once individual get jobs they will remain employed. Welfare-to-work programs are 
particularly insistent on the value of getting any kind of job, and the current political 
rhetoric in the United States about "ending welfare as we know it" concentrates on pushing 
welfare recipients into work. This tactic assumes that there are plenty of jobs available to 
those who want to work and that the appropriate motivation to work — from either the 
"stick" of reduced welfare benefits or the "carrot" of increased incentives to work — will 
be sufficient. There has been much less attention paid to the problem of enhancing the basic 
competencies — cognitive, vocational, and personal — of job trainees, except in a limited 
number of intensive and experimental programs. 49 

The success of this strategy is confirmed in the widespread finding that job training 
programs increase earnings by increasing the amount of employment, rather than by 
increasing fHe wage rates (and presumably the productivity) of individuals. But. this 
strategy ignores the reality that the low-skilled labor market for which job training 
programs prepare individuals is so unstable that — without the increase in basic skills that 
would enable individuals to escape the secondary labor market — they will continue to 
suffer intermittent employment, low earnings, and the kind of discouragement that leads 
them over the long run back to marginal employment or welfare. (This is consistent with 
the finding in Table 18 of benefits declining in years 4 and 5.) Furthermore, the kinds of 

49 The current evaluation of the JOBS program in 7 states will provide a test of the "gct-a-job" approach 
versus skills training, since individuals wUl be randomly assigned to the two different service approaches. 
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jobs that individuals leaving such programs can typically get are so dreadful — with 
repetitive, boring work, few prospects for advancement, and often harsh and demeaning 
supervisions — that it is no wonder that individuals leave after short periods of time. 
Ethnographic and journalistic accounts have sometimes stressed the difficulty of job- 
keeping rather than }6b-finding (Quint, Musick, and Ladner, 1994), and a current 
experiment is intended to assess the value of continuous services intended to help 
individuals keep the jobs they find. But whether this is possible without changing the 
nature of low-wage work is an open question. 

The implication of this argument is that, in the interests of greater long-run effects, 
there should be more attention to the enhancements of skills — both basic education and 
job skills — and less attention simply to getting individuals into employment. 50 Indeed, the 
long-run evidence on the effects of welfare-to-work programs, summarized in Section in.6 
and Table 18, tends to confirm this: the only program without long-run decay in earnings 
was the Baltimore Options program, which was distinguished from the others by more 
intensive education and training (Friedlander and Burtless, 1995, p. 144). The most 
powerful evidence, however, is the contrast between the typical job training benefits — 
which decay over 4 or 5 years — and the age-earnings profiles associated with different 
levels of education, where the benefits of education expand over time. 

3. The poor quality of job-related training 

When they do not emphasize pushing individuals into employment, job training 
programs sometimes provide (as their name implies) some training in job-specific skills. 
Sometimes this takes place in classroom settings, and sometimes in work settings or on- 

50 There is even a small amount of evidence for this proposition. In Fischer's (1995) meta-analysis, the 
effects of job search increase in quarter 2 and then decay essentially to zero in quarter 4, while the effects of 
basic education programs are initially negative but then increase through quarters 2, 3, and 4. In addition, 
the effects of "staged" job search — in which individuals are screened through job search and then proceed to 
other activities including education — also increase, presumably both from screening and human capital 
effects. See Table B.l. y 
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the-job training. 51 However, a study of on-the-job training revealed that in a large fraction 
of these programs (55 percent), there was little or no explicit training going on: employers 
viewed the program as a source of subsidized labor and used individuals in routine, 
unskilled work, without any attention to providing either job-specific or more general skills 
(Kogan et aL 1989). This approach among employers to on-the-job "training", which 
occurs in a variety of job training and apprenticeship programs, 52 is particularly likely io 
occur where employers are small and marginal and are pressed for resources. In many local 
programs, JTPA agencies seem to act as a screen to provide such employers with a steady 
source of relatively stable, low-cost labor, and can therefore come up with jobs for JTPA 
trainees — but these placements have very little training and few long-run prospects. 

The quality of classroom-based job skills instruction in job training programs has 
not, to my knowledge, ever been closely examined. However, here too there are likely to 
be serious problems. Keeping up with technological changes is difficult enough in the more 
sophisticated, longer-term programs offered in community college ; and technical institutes, 
but in short-term job training programs with little funding for capital outlays it must be 
nearly impossible. Similarly, the problem of finding instructors from i idustry is a difficult 
issue for postsecondary educational institutions, and this problem must even more serious 
in local job training programs with their intermittent offerings and therefore unstable 
employment of instructors. Many job training programs are operated by community-based 
organizations, which typically pay low wages. And because such organizations are often 
principally involved in other activities — promoting the rights of black Americans, for 
example, or of recent immigrants, or advocating on behaif of the disabled — their 
experience in job training and education and their connections to employers may not be 

51 There is a 2ong-running debate in the United States about whether classroom-based or work-based 
training is more effective. The recent infatuation with the work-based apprenticeship systems of Germany 
and other European countries has led to greater interest in work-based learning even though the pedagogical 
problems are virtually the same in both settings, as E^rryman (1995) has pointed out. 

52 ^ or * dc . scri P l i°n of the generally excellent co-operative education programs in Cincinnati, 
distinguishing those who view co-op programs as an educational experience versus those who view it as a 
source of well-trained short-term labor, see Villenueve and Grubb (1995). 
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strong. While the quality of job-related instruction merits further investigation, the 
conditions in many job training programs are not conducive to high-quality training. 



4. The deep ignorance of good pedagogy 

In educational institutions, there is currently a great deal of debate about the most 
effective pedagogies, and a concerted effort by reformers to replace conventional, didactic 
methods of teaching — what I have called "skills and drills" — with other approaches to 
teaching associated with a very different tradition of "meaning making" that enable students 
to be more active in learning, that are student-centered rather than teacher-directed, and that 
use a wider variety of activities and motivation in the classroom. 53 In adult education too, 
there has developed an orthodoxy of good practice that advises programs to tailor 
instruction to the interests and goals of adults and to use a variety of instructional methods 
including more active techniques. Unfortunately, none of this discussion has affected the 
world of job training programs, where even the existence of a debate about pedagogy is 
unknown. Job training programs virtually universally use conventional pedagogical 
techniques based on "skills and drills", with instructors breaking reading, writing, and 
mathematical skills into tiny sub-skills and drilling endlessly on a series of inherently 
meaningless sub-skills (Grubb et al., 1992; Grubb and Kalman, 1994). 54 The instruction 
is particularly bad in programs that have adopted computer-based instruction: while 
administrators are often quite proud of their computer programs, the existing programs are 
the worst examples of "skills and drills" converted to the computer screen, with even 
shorter reading passages, less writing, and more trivialized arithmetic examples than 
standard textbook instruction (Weisberg, 1988). Here too there is often a pride among job 

The issues in "skills and drills" versus "meaning making" as approaches to teaching arc complex, since 
each involves many different assumptions about the nature of learning, the roles of student and teachers, the 
appropriate competencies to be taught, and the like; for summaries to these differences see Grubb et al 
( 1 992) and Grubb and Kalman ( 1 994). 

54 There are almost surely a few exceptions since the job training world is so large and varied; for a very 
brief description of one of them, in San Diego, see Martinson and Friedlander (1994V 
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training administrators in distinguishing themselves from what educators do: they will say, 
for example, that they are "trainers" rather than "educators", and those in charge of 
computer-based programs well describe themselves as "managers" of the program rather 
than "teachers". But this pride masks their deep ignorance of pedagogical issues, and 
results in instruction that is quite horrifying to see. 55 

While there is evidence that conventional didactic approaches are the least effective 
methods for teaching many individuals, 56 this approach is likely to be particularly 
ineffective for the individuals in job training programs. Most of them by construction have 
not done well in many years of conventional schooling, using conventional didactic 
instruction; why they should suddenly be able to learn from this approach in very short 
programs with bad teaching is completely unclear. The ineffectiveness of conventional 
approaches to teaching may be inferred by a study of remedial education in the GAIN 
program: the only county with an increase in test scores was San Diego, which developed 
an innovative program to avoid the problems with the "school-like" adult education system. 
As one administrator described their efforts: "These people had an unproductive experience 
in school and were not able to benefit. We wanted to avoid the perception that they were 
going back. We wanted to make it different and make it work for them (Martinson and 
Friedlander, 1994, p. 41). 



^ In the course of observing job training and remediation programs, we saw several — in particular a 
STEP program — that almost caused us to violate good research protocol by complaining to senior 
administrators about the cruel and unusual practices we observed. Even in the highly-regarded CET program 
the teaching was completely conventional. 

56 Some direct evidence based on learning outcomes of the superiority of alternatives to conventional 
teaching is available for elementary students (Knapp, Shields, and Turnbull, 1992); a meta-analysis of 
writing has shown that the presentational (or didactic) mode and the conventional teaching of grammar are 
the least effective approaches (Hillocks, 1986); and some specific practices have been confirmed superior, 
like cooperative learning (Slavin, 1987). However, there is relatively little evidence based on learning 
outcomes for adults taught in different ways partly because there is relatively little empirical research of any 
kind about adult education, developmental education in community colleges, and basic skills within job 
training programs and partly because the efforts to evaluate outcomes of different instructional methods have 
used inconsistent conceptions of instruction and therefore inconsistent observations of classrooms; see 
Romberg and Carpenter (1986) for lnath and see Hillocks (1986)for writing. Another problem is that 
different approaches to teaching generally emphasize different goals: Advocates of teaching in the meaning- 
making tradition usually stress "authentic" tasks and higher-order competencies, notoriously difficult to 
assess reliably, while those following skills and drill are more likely to be content with standardized tests. 
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The inability or outright refusal of many job training programs to understand 
pedagogical issues is exacerbated by the problems of hiring instructors. There has been 
little research about those who teach in job training programs; but the conditions of 
providing short-term and intermittent programs, often in community-based organizations 
with low pay, are not conducive to hiring good teachers. Typically, instructors in job 
training programs are given little preparation in teaching — a further indication of how little 
attention is given to teaching. In contrast, within the schooling system in this country the 
issue of preparing teachers well, and of paying them enough to attract a stable, 
experienced, and dedicated teaching force, are widely discussed. The fact that job training 
programs have typically not even raised this question is yet another sign of the 
unimportance of teaching — and another contributor to the low quality of instruction. 

5. Local political interference 

Many job training programs, particularly those operated by JTPA <ind JOBS (which 
often works through JTPA programs), are highly local. A local decision-making authority, 
the Private Industry Council or PIC, makes important decisions about the nature of services 
to be provided, to which groups of individuals, and then establishes the methods for 
subcontracting with other groups (typically, community-based organizations, educational 
institutions, and proprietary schools) to provide the various services. The local nature of 
these decisions is certainly necessary in light of the fact that low- and middle-skilled labor 
markets are themselves quite local, and therefore programs must adjust themselves to local 
conditions. However, this also make job training programs vulnerable to local political 
influence as well. This usually operates to direct funds to particular providers of services, 
regardless of whether they are effective or not, and makes it difficult for local programs to 
shift resources from ineffective providers to more effective organizations. 



Political interference appears to take place in several different ways. 57 In some 
cases, providers of training services are represented on the PIC, and seem to be able to 
direct contracts to their own organizations. In other cases, influential local politicians can 
effectively threaten to create trouble for a job training program if they do not support a 
favorite local provider. In still other cases a local job training administration, anticipating 
political problems, will arrange to allocate resources through non-competitive processes 58 
which are essentially rigged so that particular community-based organizations receive 
funding. In many cases, this kind of interference takes place on behalf of certain groups 
with particular racial identities — for example, a local group representing the black 
community or the Hispanic community (or particular parts of the Hispanic community like 
Haitians or Puerto Ricans). In other cases the community-based organizations represent 
women, or older women trying to re-enter the workforce, or the handicapped, or some 
ether particular group. They operate simultaneously as advocates for "their" groups, as 
sources of guidance and counseling for members of these groups, and as providers of 
education and training services through public funding. In all these cases, constituents 
served by such CBOs can exert considerable political pressure. Unfortunately, 'iome of the 
worst job training programs seem to take place in cities with well-organized minority 
community-based organizations with political influence; in these cases the difficulty of 
detecting political interference is compounded by racial tensions, which make it difficult for 



^' The question of local political interference is widely acknowledged by observers of job training 
programs, but almost no one has acknowledged the problem in writing — probably because this problem 
has racial dimensions that are highly controversial in the United States; for an attempt to describe the 
problem in one community (Fresno) see Grubb and McDonnell (1991). Other efforts to cope with this 
problem arise in cases where particular JTPA programs have been investigated for fraud, mismanagement, 
or ineffectiveness. There has been to my knowledge no real analysis of the local politics of job training 
programs, however. 

It is common in the job training world for local programs to use a competitive RFP (request for 
proposal) process where the SDA invites organizations to bid on proposals for providing services; the 
competitive RFP process is intended to increase the number of organizations bidding on the basis of low 
cost and high quality. However, there appear to be a number of ways to manipulate the RFP process so that 
certain organizations will be chosen; for example, RFPs can be written so they apply only to specific 
organizations, or the knowledge that an RFP is rigged for a particular organization will prevent others from 
applying. One individual commenting on an early draft of this monograph reported that a charismatic PIC 
director had his life threatened for changing contracting procedures. 
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white administrators at the local or state level to challenge the allocations of funds to 
ineffective black or Hispanic groups. Conversely many of the best programs seem to take 
place in rural and suburban areas that are relatively free of such political interference. 59 

The effects of local political interference on the effectiveness of job training 
programs is difficult to assess. In many cases, it is clear that ineffective organizations are 
given resources because of political interference, and effectiveness suffers. However, 
community-based organizations with political influence also include many extremely 
effective organizations, highly dedicated to the group they serve. My own hunch is that — 
as is generally true of the private sector compared to publicly-provided programs — the 
community-based organizations providing job training services include some of the best as 
well as some of the worst providers. The problem with local political influence is that it 
becomes difficult to eliminate some of the worst providers — and those who enroll in them 
are the ones who suffer. 

6. Low placement rates 

One of the conventions in vocational education and job training is that the labor 
market value of job-specific education and training is likely to be quite low if individuals 
are unable to find jobs for which they have been prepared. While there has been relatively 
little analysis of the consequences of job-related versus unrelated education and training, 
the little evidence that exists indicates that job-related vocational education does have higher 
economic benefits than unrelated education (Grubb, 1995b; Rumberger and Daymont, 
1984). 

In the world of job training, some services convey general competencies — for 
example, remediation should enhance basic academic skills that are useful in virtually every 



59 



For example, in a study of efforts to coordinate job training, welfare- to-work programs, and vocational 
education, none of the exemplary coordination efforts were in cities because of the dominance of purely 
political allocation of resources there; see Grubb et al. (1991). 



83 



job, and on-the-job training and work experience that enhance the personal characteristics 
required at work should have the same effects. But a good deal of job skill training and on- 
the-job training is job-specific, and may not benefit individuals much if they fail to find 
related employment. In one particular data' set — the Survey of Income and Program 
Participation (SIPP) — individuals are asked if they have received different forms of job 
training, and whether this training is related to their current employment. Among those who 
were in JTPA programs, 49 percent of men and 46 percent of women reported that they 
used training on their current job; for those reporting having enrolled in CETA, 42 percent 
of men and 46 percent of women reported that their job training was related (Grubb, 
1995b). And — consistent with the hypothesis that related training has a much higher 
economic return — the earnings of men with related JTPA training were on the average 55 
percent higher than those with unrelated training, with the comparable figure for women 42 
percent; for those with CETA training, men earned 21 more if their training was related to 
their current job, while women earned 6 percent more. 60 Thus the average economic 
benefit of job training programs — the figures that show up in formal evaluations — are as 
low as they are partly because they combine the much lower benefits (presumably near 
zero) of those who failed to find related employment with the more substantial benefits of 
those with employment related to their job training. The conclusion in the SIPP data that 
only a minority of individuals are in jobs related to previous job training suggests that, even 
with some placement efforts, job training programs have not done a good job of placement. 
Alternatively, if individuals are placed in related jobs with little future, then they would 
normally shift over time into different occupational areas with greater prospects for 
mobility. Under either explanation, however, job training programs appear to do a 
mediocre job of placing individuals in appropriate jobs. 

60 These are results for calendar year 1987. Of course, the quality of the data entering these calculations is 
much poorer than the data in random assignment experiments: the SIPP data arc entirely self-reported and 
arc retrospective, generating problems of whether the reports of having been enrolled in job training arc 
accurate, as well as assessments of job-rclatedncss. The other independent variables available in the SIPP 
data are inadequate to control for the various characteristics of those in job training programs; therefore all 
the coefficients on JTPA and CETA job training programs arc negative, reflecting negative selection. 
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7. "One-shot" job training and the links to other programs 

One of the basic characteristics of job training programs in the United States is that, 
since they enroll individuals with substantial barriers to employment and provide relatively 
limited training, they aim to place individuals in jobs with relatively low levels of skill and 
pay. The limited ambitions of these programs — confirmed by the finding that increases in 
earnings are modest, and probably decay after 4 or 5 years — may be "realistic" given their 
resources, but they still do not help individuals move out of poverty or off welfare. And the 
emphasis on quick placement into employment, while it generates modest benefits in the 
short run, reinforces the notion that job training should be a short, one-shot event, with 
individuals then leaving the job training system. 

Typically, job training programs are not linked to other programs, either to other 
job training programs or to education programs. 61 On occasion, job :raining programs refer 
individuals to other programs — for example, they may refer those in need of remediation 
to adult education programs — but in these cases there is little effort to identify which such 
programs might be most effective, or to follow individuals to make sure they enroll and 
complete other programs (Grubb et al., 1991). Even in welfare-to-work programs, where 
there is a caseworker assigned to each individual to make sure that he or she can navigate 
the array of services offered, individuals often become "lost" when they are referred to 
services but then never enroll, or never complete the program, or fail to enroll in 
subsequent programs. As one GAIN administrator in California commented about 
individuals referred for remedial education, the lack of information about progress means 
that many clients "fall into the black hole of adult basic education", staying in adult 
education for long periods of time without much progress and without caseworkers 
knowing where they are (Grubb and Kalman, 1994). When individuals complete job 

6! On the problems in the current "system" of education and training, sec especially Hansen (1994); Grubb 
and McDonnell (1992); McDonnell and Grubb (1991); Grubb et al. (1992); and Grubb et al. (1991). 
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training programs they are usually referred to employment, not to subsequent education; 
and of course the need for most trainees to earn a living generally precludes them from 
immediate enrollment in other education or training. The consequence is that the 
possibilities for expanding resources to those in job training programs — for gaining them 
access to additional training that might provide them with access to jobs of increasing skills 
levels and pay — are virtually non-existent, 62 and it is not surprising that the benefits of 
job training seem to decay over time. 

But an alternative way to view education and training is the notion — sometimes 
encapsulated in the overworked phrase "lifelong learning" — that a low-level job training 
program would be only a start back into the labor market. If job training programs were 
linked to one another and to education programs, then an individual could enroll in a low- 
level program, complete it and enter low-skilled and low-paid employment, and then — at 
such a point where tine and resources permitted — could continue in a more advanced job 
training program, or a credential program in a community college, in order to gain access to 
higher-skilled and better paying jobs. This kind of "ladder" of training and education 
opportunities would therefore be able to take an individual at any level of skill and then — 
in a series of short-term programs rather than a single, one-shot program — provide access 
to a range of jobs with better long-run prospects, a possibility to which I turn in the 
Conclusion. 63 



8. Labor market explanations 



b/ There are only a few exceptions to the general pattern of job training occurring in isolation from other 
education and training programs. In some welfare-to-work programs, caseworkers emphasize "self-initiated 
placement", where an individual can put together a series of individualized education and training programs. 
And in some cases job training programs contract with community colleges to provide short-term job 
training; in such cases administrators claim one of the benefits to be that trainees can more easily enroll in 
regular community college programs — though the frequency of movement from short-term job training to 
longer-term educational programs in such situations is unknown. 

° J Part of the history of education in the United States is the development of a coherent system, with 
clearly-articulated links among different levels of education — though this system was not complete until 
the 1920s or so. One could argue that the job training "system * is still in its infancy, since it is barely 30 
years old — and that it will take a longer period of time for a system to emerge. 
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A possible explanation for the mediocre employment effects of job training 
programs is simply that there are not enough jobs for unskilled and semi-skilled workers, 
and the labor market is unable to absorb those who complete job training programs. 64 For 
example, the finding that welfare-to-work programs in West Virginia were the least 
effective (see Table 8) was generally attributed to weak economic conditions in those 
regions. However, in other cases it has proved impossible to blame labor markets. For 
example, the outcomes across the six GAIN counties of California could not be explained 
by variation in labor market conditions; and the analysis of JTPA after 18 months found no 
significant effect of the local unemployment rate on earnings, and only a minimally 
significant effect of urban location on youth (but not adult) outcomes (Bloom et al., 1993, 
Exhibit 7.12). In addition, as I pointed out in Section n, labor market conditions have 
contradictory effects: although high unemployment or low employment growth may make 
job placement more difficult, it may also cause more job-ready individuals to enroll in job 
training programs, making placement easier than in boom times when those enrolled in job 
training programs have the greatest barriers to employment. In general labor market 
explanations have not been popular among those examining job training programs. 

However, it is still possible that labor market conditions may explain the mediocre 
effects of job training programs. In the first place, there have been relatively few systematic 
examinations of labor market effects, aside from that undertaken for the JTPA evaluation, 
and the examination of labor market effects on JTPA may have been marred by insufficient 
variation in labor market conditions. 65 Second, it is possible that the weak condition of 

64 Equivalently, it may be that individuals completing job training programs find employment at the 
expense of others who do not — though, as mentioned in Section II, this kind of displacement cannot be 
detected with conventional evaluation methods. In economic terms, a shift outward in the supply function 
for a particular kind of labor (e.g., for modestly-skilled employment) along a stationary and inelastic demand 
function will result in very little additional employment and in a fall in the wage rate — and so placement 
rates will be low, displacement high, and the increase in earnings modest. In effect, job training programs 
assume that the demand for labor is relatively elastic. 

65 Because there were only 16 sites, there was not much variation in labor market conditions. In addition, 
the results were carried out only for the 18-month outcomes, not the 30-month results. 
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labor markets for modestly-skilled work explains the pervasively mediocre results of job 
training programs, even if it fails to explain the cross-section variation in outcomes. 
Finally, it is worth noting that job training programs represent a supply-side solution to the 
problem of underemployment and poverty — the assumption that if the skills of the labor 
force can be improved, or if individuals out of the labor force can be induced to enter it, 
then employment and earnings will improve without intervention on the demand side. The 
alternative tactic, of course, is that a supply-side policy should be coordinated with a 
demand-side policy to increase the demand for modestly skilled workers. Indeed, 
something like this took place in the last years of the CETA program, when public service 
employment (PSE) created additional jobs for CETA trainees in governmental and non- 
profit community-based organizations. Such efforts were denounced as "un-American" 
since they might substitute public employment for private employment, and were quickly 
abolished by the Reagan administration. But the notion that demand-side policy in labor 
markets should be coordinated with the more common supply-side policy remains an 
attractive idea, and one that might improve the mediocre effects of job training programs. 

9. The special problems of youth programs 

Over i!;e past two decades, the results of evaluation youth programs have been 
especially dismal. Many of the early CETA programs for youth had negative results (see 
Table 3), with the exception of the expensive Job Corps program (T -ble 4), and the 
minimal or negative effects were replicated in the more sophisticated JTPA evaluation 
(Table 5 - 7). Several of the experimental programs for youth, like JOBSTART and STEP, 
have proved ineffective despite careful planning and higher costs. These results are 
particularly discouraging because of the hope that programs aimed at youth could be 
preventive, by steering young people into paths that would be beneficial in later years. 
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There are several explanations for the particularly poor results in youth programs. 
One has to do with labor market conditions: many employers will not hire young people, so 
under the best of conditions they have a tendency to mill around until reaching their early or 
mid-twenties. In addition, for many of the moderately-skilled jobs in the sub-baccalaureate 
labor market, employers will not consider individuals without a high school diploma, 
effectively condemning drop-outs to completely unskilled positions. Other explanations 
depend on the special characteristics of youth culture in the United States: this culture, with 
its rejection of school and discipline and the premium placed on "coolness", may work 
against job training training programs in ways that do not affect the performance of adults 
with greater maturity and sense of responsibility. Still other explanations point out that 
adolescents are still entangled in their families, which may be disorganized and destructive 
rather than supportive (Quint, Musick, and Ladner, 1994). 

Unfortunately, job training programs cannot do much about these factors, 
However, there may be some systematic failings in the job training programs applied to 
young people (Granger, 1994; Doolittle, undated). Programs devised for adults may not be 
developmentally appropriate for adolescents; and the conventional "skills and drills" 
pedagogy of most job training may be especially abhorrent to them, because of their recent 
and negative experiences with school. Because of the complex conditions of their lives, 
young people may need a greater variety of support services — especially guidance and 
counseling about such issues as drugs, alcohol, sex, and sexually-transmitted diseases — 
thai are not usually part of job training programs designed for adults. To be sure, the 
minimal results of programs designed specifically with a range of services — like New 
Chance and JOBSTART — are not especially encouraging, but most of the efforts to 
devise better youth programs still concentrate on specifying a broader array of services to 
cope with the more complex conditions of adolescents' lives. This is essentially the vision I 
offer in the Conclusion, in which work-based placements would be combined with school- 
based activities (as well as a variety of supportive services) to provide a variety of learning 



opportunities for those individuals who have not done well in conventional educational 
institutions of the "first chance" system. 

10. The impossibility of "second chance" programs 

A final possibility is that the very idea of providing "second chance" programs is 
flawed. Given the universality of "first-chance" programs — the elementary-secondary 
education system in the United States, with its many remedial and compensatory programs, 
along with a higher education system that virtually guarantees a place for everyone (in 
open-access community colleges if not in four-year colleges) — one could argue that the 
individuals who fail to use the educational system to increase their skills and gain access to 
employment are by definition those with such serious intellectual, personal, and 
motivational barriers to employment that no "second chance" system of reasonable cost 
could possibly help them enter stable employment. Under this argument, the resource:; that 
currently go into job training programs should be diverted into improving the "first chance" 
programs; alternatively, one could argue (as some have 66 ) that there has been too much 
spent already within the first-chance educational progiams on preventing school failure, 
and the education and training system should devote less of its resources to the bottom 
tenth, or those that have sometimes been demeaned as the "leftovers" (Taggart , 1981). 

But this view — the abandonment of second-chance opportunities, though job 
training programs and such institutions as adult education and community colleges — is a 
distinctly un-American idea. Even now, when the country has swung toward the political 
right, with many proposals for dismantling important aspects of our welfare and regulatory 
systems, there are almost no calls to dismantle the education and training system. One 
reason is that even the most ineffective efforts at training for employment are more palatable 
than the alternative — allowing individuals to live at public expense without working. And 

66 See, for example, Traub (1995), who complains about the inappropriateness of providing so much 
remediation in colleges. 
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the effort to build second-chance programs is an expression of the enduring American 
commitment to equity in some form — even if that form is not particularly effective. For 
these reasons this country is unlikely to abandon its current efforts at job training. Daunting 
as it may be, the appropriate task is that of improving job training programs, rather than 
abolishing them. 
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V. Conclusion: A Vision for Successful Job Training Programs 



How might job training programs be improved? Drawing on the explanations in the 
previous section, one tactic might be to improve each of their components. That is, the 
quality of job training needs to be carefully considered, and should be improved in many 
programs; the nature of instruction in basic skills is often very poor, and job training 
programs need to learn more from the education system about appropriate instructional 
methods; and efforts in assessment, case management, and placement may need to be 
strengthened (e.g., Dickinson, Kogan, and Means, 1994; Dickinson et al., 1993). 

But this kind of piecemeal approach, valuable though it might be in specific 
instances, seems to miss the point. The real problem with existing job training programs is 
not that an individual component here or there is not of adequate quality, but that the 
offerings of the "system" as a whole consist of a welter of different services, none of them 
obviously more effective than any others and all of them poorly coordinated, with 
individual programs of limited intensity not linked to other opportunities even though they 
are intended for a population with substantial needs. In contrast, the most effective 
programs — the Center for Employment Training and the Riverside GAIN program, for 
example — seem to work because they encompass a combination of mutually-supporting 
practices. This suggests that the most powerful approaches to reforming job training would 
first create models of more comprehensive and interactive employment-related services, and 
then worry about the quality of individual components. 

Furthermore, it would be important to connect job training programs to other 
training and education opportunities, rather than leaving them as independent, limited, one- 
shot efforts. The effects of job training programs, small as they are, seem to decay over 
time, while the benefits of education typically increase with further labor market 
experience. The disconnection between "education" and "job training", rooted in the very 
creation of job training programs during the 1960s, has been counter-productive for both 
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Many of the reasons I gave in the previous section about the ineffectiveness of job training 
came from this divorce — including the small scale of job training efforts, the ineffective 
pedagogy, the political influences in job training, and the provision of services in small, 
unstable organizations. And conversely education institutions could learn much from job 
training programs about the importance of employment and of services like job placement. 

Fortunately, there is already a model in the United States that could be used as the 
basis for reforming job training, though it has not yet been implemented. The School-to- 
Work Opportunities Act, passed in May 1994, is intended to apply to secondary and 
postsecondary education programs, but it presents a vision that could be used to guide job 
training programs as well. The STWOA can be interpreted as specifying five elements for 
successful programs: 

1. The inclusion of academic instruction. In many federal programs like JTPA and 
JOBS, these academic components are either remedial education or English as a Second 
Language (ESL). 

2. The inclusion of vocational skills training, integrated with academic (or remedial) 
instruction. Integration does not imply (as it does in job training) that individuals receive 
both kinds of instruction, at different times of day; it is a much more complex practice in 
which academic and occupational content are combined within a single class, sometimes 
with the collaboration of two different instructors (Grubb, 1995a). 

3. The inclusion of work-based education, coordinated with school-based 
instruction through "connecting activities", to provide a different kind of learning (as the 
Center for Employment Training provides). 

4. The connection of every program to the next program in a hierarchy of education 
and training opportunities. In the STWOA, high school programs are explicitly linked to 
postsecondary opportunities. The analogy in job training programs is that every program 
would be connected to further programs providing higher level of skill and access to 
enhanced employment opportunities. 
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5. The use of applied teaching methods and team-teaching strategies. By 
implication, all school-based and work-based instruction should develop pedagogies that 
are more contextualized, student-centered, active (or constructivist), and project- or 
activity-based; they would adhere to the standards of good practice that have been 
developed for adult education, rather than ignoring pedagogical issues as job training 
programs currently do. 

This vision of job training programs would combine the resources currently 
available in different federal programs to create more integrated efforts. 67 These would 
combine, for example, remediation currently funded by the Adult Education Act, the Adult 
Literacy Act, JTPA and JOBS; vocational skills training supported by Perkins, JTPA, and 
(sometimes) JOBS; on-the-job training and work-based experience now funded by JOBS, 
and potentially the STWOA; support services funded by JTPA and JOBS; and income 
maintenance available to welfare recipients in JOBS and to students through Pell Grants. 

Such programs would also try to establish links among programs in order to create 
education and training "ladders" — that is, a series of sequential education and training- 
related activities that individuals can use to progress from relatively low levels of skill (and 
relatively unskilled and poorly-paid work) to higher levels of skill and (presumably) more 
demanding, better-paid, and more stable occupations. 68 Individuals could enter the system 
at any level — for example, welfare recipients with the lowest levels of education and labor 
market experience could enter programs similar to those now available through JOBS, but 
each program would be linked to other programs further up in the hierarchy of skills. The 
programs in the current job training system would be articulated with those in the education 
system through the community college — that is, certain job training programs would lead 

67 For a primer recommending practices for summer youth programs that is quite similar to this 
recommendation, see Center for Human Resources (1993). 

68 The ideas in the section draw upon Grubb (1992). Within the social services field, the same idea is 
known as providing a "continuum of services" — for example, providing a range of mental health programs 
from highly restrictive, for the most dangerous and worst-off patients, to minimum security facilities and 
halfway houses, to various forms of counseling and therapy for those able to live on their own. In theory, 
individuals can enter the continuum at any point appropriate to their needs, and progress up and out of the 
system. 
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into certificate and Associate programs in the community college, from which individuals 
could over time move into baccalaureate programs in four-year colleges. In this way 
individuals with minimal skills could move up a "ladder" of ever-expanding opportunities. 

There are several reasons for emphasizing vertical integration and the creation of 
education and training ladders, in place of isolated job training programs: 

• The individuals within the education and training system who are in the greatest 
need require a number of different services: basic language skills and other competencies; 
job-specific skills; personal attributes like motivation, discipline, and persistence; help in 
conquering drug and alcohol dependencies, or mental health problems; and decision- 
making skills, making it difficult for them to negotiate programs on their own and requiring 
a counselor or caseworker. They need so much that it's impossible to think about 
integrating them into the economic mainstream except in small steps, providing support (via 
welfare, training subsidies, or unskilled work) in the interim. It might, of course, be 
possible to develop much more comprehensive and intensive programs like Job Corps, but 
these are so expensive that they are politically difficult to provide; a linked series of small 
programs is both more flexible and more acceptable politically. 

• Many of those in need of education and training need to work while they are 
enrolled in programs. This is true almost by definition for those seeking retraining, upgrade 
training, and second-chance training, but it is also true for a number of those seeking initial 
entry in to the labor force. This means that they cannot afford to stop working for two to 
four years to accumulate a credential; they need instead to accumulate small amounts of 
education and training, work a while, and return to school (probably part-time) to further 
improve their skills. 

• Even where communities provide a range of services, the pathways through them 
are unclear, particularly to those who are unsophisticated about finding and using 
programs. Furthermore, eligibility standards are inconsistent, assessment procedures are 
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varied, and the content of programs is uncoordinated — and so in practice the range of 

♦ 

education and services provided is not a smooth continuum. 

• The labor market has certain barriers which can be overcome only through specific 
education credentials. The labor market for middle-skilled occupations is almost completely 
closed to individuals without high school diplomas; therefore relatively unskilled 
individuals (e.g., high school dropouts in JTPA or JOBS programs) have a crack at entry- 
level jobs in the middle-skilled labor market only after they get their diploma. With 
experience, ability, and motivation they can advance, but some occupations may require 
subsequent education (e.g., from community colleges or technical institutes, in computer 
applications, specific business procedures, CAD, specific electronics training, etc.) in order 
to advance. Then many jobs — management positions, accountants, computer 
programmers, many health positions, and most professional jobs with real advancement — 
require B.A.'s. That is, advancement can be blocked sequentially by the lack of the high 
school diploma, certain forms of sub-B.A. education, or the B.A. itself — so that 
continuous advancement requires going back into the education and training system to 
obtain further skills and credentials. This is particularly true in cases (like health 
occupations) where these skills have been codified by occupational licensing; and if the 
country moves toward skill standards in other occupations, this will be institutionalized in 
other occupations. 

• Existing programs are hierarchically arranged anyway — this is one reason that 
there isn't that much duplication. The clients in AFDC are typically less well-prepared and 
experienced than those in JTPA, who in turn are less well-prepared than the typical 
community college student. In job skills training, job training programs provide shorter and 
less sophisticated programs than community colleges. In the literacy/remediation world, 
community-based volunteer programs are set up to take in those who are completely non- 
literate, while adult ed programs typically start at the 4th or 5th grade-level equivalent and 
lead to a GED, while community college programs probably don't start at that low a level 
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but aim to prepare individuals for college-level English courses. As long as there are such 
hierarchies in existing programs, it makes sense to take advantage of them. 

• Currently there are no mechanisms for revealing low quality. If programs are 
linked vertically, and a receiving program finds preparation in the sending program to be 
inadequate, there is a new incentive for blowing the whistle. 

• The creation of "ladders" of vertically-integrated education and job training 
programs would make it possible to move individuals into a succession of jobs of ever- 
increasing skill and earnings — eliminating the kind of "fade-out" of earnings gains typical 
of training programs. 

The realization of this vision for an education and training system, in which all 
components are related to one another, might seem to be a daunting task. Implementing 
such a system would take substantial time and legislative care — qualities that are in short 
supply in the legislative system of the United States — as well as additional public funding, 
which seems unlikely given the current anti-spending mood of the country. But as an 
overall vision for job training, it has the potential for eliminating many of the problems 
which beset current job training programs and for improving the outcomes which now 
seem so meager — in ways that a piecemeal approach to reforming job training could never 
accomplish. 
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Figure 1 

Effects of CETA Programs Over Time 



Average Annual Earnings for Male CETA Participants 
and Comparison Group Members from 1964 to 1978 
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Average Annual Earnings for Female CETA Participants 
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Source: Bloom and McLaughlin (1982), Summary Figures 1 and 2. 



Figure 2 



Effects over Time on Earnings and AFDC Payments 
of Welfare-to-Work Programs 



Year-by-Year Impacts on Earnings 
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Source: Friedlander and Burtless (1995), p. 17. 



Table 1 

Federal Employment and Training Programs by Agency, Fiscal Year 1995 



Department and Program Fiscal year 1995 Appropriation 
(in millions of dollars) 

Department of Labor 

JTPA $4912.5 

Veterans programs 175.1 

Employment service 845 .9 

Other 910.5 

Department of Education 

Vocational Education 1 236.2 

Adult Education 270.6 

Literacy programs 38.5 

Student grants and loans 47 1 6.0 

Rehabilitation services 2086. 1 

Other 638.0 

Department of Health and Human Services 

JOBS 1300.0 

Community Services Block Grants 426.3 

Refugee assistance 105.0 

Other 192.3 

Department of Agriculture 

Food stamp employment and training 165.0 

Department of Housing and Urban Development 

Youthbuild 50.0 

Empowerment Zone and 640.0 

Enterprise Community Program 

Other 47.3 

Other departments 1763.3 

Total $20,413.9 



Source: U.S. G.A.O. (1995), Appendix II. 
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Table 2 



Annual Estimated Earnings Increases for Participants 
in pre-CETA Remedial Education and Job Training Programs 



Classroom Training 




On-the-Job Training 




Asnenielter ( 1 y/o) 




Cooley, McGuire, 




Black males 


$318 to 417 


and Prescott (1975) 




\t TV 'a 1 

White males 


139 to 322 


Males 


$-38 to $59 


Black females 


441 to 552 


Females 


30 to 226 


White females 


354 to 572 










Ketron (1979) 




Borus (1964) 




Minority males 


1984 


Males 


305 


White males 


2181 






Minority females 


884 


Borus and Prescott 




White females 


926 


(1974) 








Males 


516 


Kiefer (1976) 




Females 


38 


Black males 


-160 






White males 


-61 


Cooley, McGuire, and 




Black females 


386 


Prescott (1975) 




White females 


926 


Males 


71 to 234 






Females 


168 to 291 


Prescott and 




Hardin and Borus 




Cooley (1972) 




251 


Males 


796 


(1971) 










Sewell (1971) 




Ketron (1979) 




Males 


375 


Minority females 


184 


Females 


754 


White females 


701 










Job Corps 




Kiefer (1976) 






Black males 


-742 to -355 


Kiefer (1976) 




White males 


-644 to -375 


Black males 


-179 


Black females 


591 


White males 


-74 


White females 


639 


Black females 


-188 






White females 


-780 


Main (1968) 


409 








Mallar(1978) 




?age; Gooding (1964; 


446 


Males 


187 


1962) 




Females without 


565 






children 




Prescott and Cooley 




Females with 


-206 


(1972) 




children 


Males 












Adult Basic Education 




Sewell(1971) 


432 








Brazzie(1966) 




Cain and Stromsdorfer 




Males 


2368 


(1968) 






White males 


828 


Roomkin (1973) 




White females 


336 


Males 


318 






Females 


12 



Source: Taggart (1981), Table 3.1. 
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Table 4 

Employment and Earnings of 1977 Corpsmembers and Controls 
Two Years After Termination 

Job Corps Comparison 

Participants Group Difference 

Fraction of time employed 

0-6 months .356 .354 +.012 

6-12 months .463 A \2 +.051 

12-18 months .549 .471 +.078 

18-24 months .543 .453 +.090 

Fraction of time in labor force 

0-6 months .682 .653 +.029 

6-12 months .716 .682 +.034 

12-18 months .747 .701 +.046 

18-24 months .763 .715 +.048 

Average hours worked per week 

0-6 months 14.46 13.13 +1.33 

6-12 months 17.82 15.84 +1.98 

12-18 months 21.55 18.38 +3.17 

18-24 months 21.39 17.73 +3.66 

Earnings per week 

0-6 months $45.84 $43.82 $+2.02 

6-12 months 64.38 58.38 +6.00 

12-18 months 82.17 72.48 +9.69 

18-24 months 21.39 17.73 +3.66 

Earnings per hour 

0-6 months $3.17 $3.37 $-.20 

6-12 months 3.61 3.69 -.08 

12-18 months 3.81 3.94 . 13 

18-24 months 3.87 4 .16 -^29 

Source: Mallar et al. (1980), Table 3.6. 
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Table 7 

Effects of JTPA on the Arrest Rates of Youths 



Percentage arrested during 
follow-up period 



Impact per 



Treatment 
group 



Control 
group 



Assignee 



Enrollee 



Female youths 4.4% 
Male youth non-arrestees 14.1 
Male youth arrestees 43.3 



During the first follow-up period 
3.6% 0.8% 
9.6 4.5** 
42.6 0.7 



1.3% 

6.5 

1.1 



Female youths 7.0 
Male youth non-arrestees 25.8 
Male youth arrestees 59.2 



During the full follow-up period 
5.3 1.7 
18.7 7.1 
55.7 3.5 



Statistical significance: ** = 5 percent. 
Source: Bloom et al. (1994), Exhibit 1 1 . 
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Table 8 



Summary of the Impact of Welfare-to-Work Programs 
in San Diego, Baltimore, Arkansas, Virginia, and West Virginia 



Outcome 



Experimental s Controls 



Difference 



Decrease/ 
Increase 



San Diego-Applicants 

Ever employed during 1 5 months 61.0% 

Average total earnings during 1 5 $3802 
months 

Ever received AFDC payments 83.9% 
during 18 months 

Average number of months 8.13 
receiving AFDC payments during 
18 months 

Average total AFDC payments $3409 
received during 1 8 months 

Baltimore-Applicants and 
Recipients 

Ever employed during 1 2 months 51.2% 

Average total earnings during 12 $ 1 935 

months 

Ever received AFDC payments 94.9% 
during 15 months 

Average number of months 11.14 
receiving AFDC payments during 
15 months 

Average total AFDC payments $3058 
received during 15 months 

Arkansas-Applicants and 
Recipients 

Ever employed during 6 months 18.8% 

Average total earnings during 6 $29 1 
months 

Ever received AFDC payments 72.8% 
during 9 months 



55.4% 
$3102 

84.3% 

8.61 

$3697 



44.2% 
$1759 

95.1% 

11.29 

$3064 



14.0% 
$213 

75.9% 



5.6% *** 
$700*** 

-0.4% 

-0.48 * 

-$288 ** 



7.0% 



$176 

-0.2% 

-0.15 

-$6 



4.8% ** 
$78 * 

-3.1% 



+10% 
+23% 

0% 

-6% 

-8% 



+16% 
+10% 

0% 

-1% 



+34% 



+37% 



-4% 
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Average number of months 4.96 
receiving AFDC payments during 
9 months 

Average total AFDC payments $772 
received during 9 months 

Virginia-Applicants and 
Recipients 

Ever employed during 9 months 43.8% 

Average total earnings during 9 $ 1 1 1 9 
months 

Ever received AFDC payments 86.0% 
during 12 months 

Average number of months 7.75 
receiving AFDC payments during 
1 2 months 

Average total AFDC payments $ 1 923 
received during 12 months 

West Virginia-Applicants and 
Recipients 

Ever employed during 15 months 22.3% 

Average total earnings during 15 $713 
months 

Ever received AFDC payments 96.8% 
during 21 months 

Average number of months 14.26 
receiving AFDC payments during 
21 months 

Average total AFDC payments $268 1 
received during 21 months 



5.49 



$865 



40.5% 
$1038 

86.1% 

7.90 

$2007 



22.7% 
$712 

96.0% 

14.46 

$2721 



-0.53 



*** 



-$93 



*** 



$81 

-0.1% 

-0.14 

-$84 *' 



-0.4% 
$0 

0.8% 
-0.21 

-$40 



-10% 



1% 



+8% 
+8% 

0% 

-2% 

-4% 



-2% 
0% 

+ 1% 

-1% 

-1% 



Statistical significance: *** = 10 percent; ** = 5 percent; * = 1 percent. 
Source: Gueron (1987), Table 2. 
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Table 9 

Effects of Massachusetts ET Program 





ET 


Comparison 




Percent 




Participants 


Group 


Difference 


Change 


Probability of employment 


44.5% 


36.3% 


8.2%*** 


23% 


Six-month earnings 


$1,537 


$1,147 


390*** 


34% 


Duration of current AFDC spell 


10.21 


14.44 


-4.23*** 


-29% 


Average monthly AFDC grant 


$294.40 


$320 


-25.6*** 


-8% 


Statistical significance: *** = 1 percent; ** = 5 percent; * = 10 percent. 



Source: Nightingale et al. (1991), Tables 4.1, 4.2, 4.3, 4.4 and associated text. 
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Table 10 

Effects of GAIN for All Counties 



Experimentals 



Single Parents 

Earnings over 3 years $778 1 

Total AFDC payments $ 1 4 ,464 

Ever employed in year 3 39.6% 

On AFDC in last quarter 52.5% 
of third year 

Received GED or high 7.2% 
school diploma 

Received trade certificate 9.2% 

Heads of 2-parent 
families 

Earnings over 3 years $10,156 

Total AFDC payments $ 1 8 , 1 64 

Ever employed in year 3 44.6% 

On AFDC in last quarter 57.3% 
of year 3 



Controls 



$6367 
$15,426 
33.7% 
55.5% 

1.9% 

8.8% 



$9045 
$19,332 
40.1% 
57.7% 



Difference 



$1414 *** 
-$916 *** 



5.9% 



*** 



-3.0% *** 
5.4% 
0.4% 



$1111 *** 
-$1168 *** 



4.5% 



-0.5% 



*** 



Percent 
Change 



22% 
-6% 



12% 
-6% 



Statistical significance: *** = 1 percent; ** = 5 percent; * = 10 percent 
Source: Riccio, Friedlander, and Freedman (1994), Tables 1, 2, 8, 9, 2.9 



Table 1 1 

Two-year impacts of Project Independence 



Outcome and 
Follow-up Period 



Program 
Group 



Control 
Group 



Difference 



Percentage 
Change 



Total cohort 

Ever employed (%) 

Years 1-2 66.3% 

Yearl 55.4 

Year 2 53.3 

Last quarter of year 2 38.3 

Average total earnings ($) 

Years 1-2 5766 

Year 1 2548 

Year2 3219 

Ever received any AFDC 
payments (%) 

Years 1-2 88.3 

Last quarter of year 1 64.6 

Last quarter of year 2 51.2 

Average total AFDC 
payments received ($) 

Years 1-2 4028 

Yearl 2196 

Year 2 1832 

Early cohort 

Average total earnings ($) 

Years 1-2 5619 

Year 1 2499 

Year2 3120 

Year 3 3538 

Average total AFDC 
payments received ($) 

Years 1-2 4003 

Yearl 2115 

Year 2 1889 

Year 3 1438 



63.8% 

52.6 
51.2 

37.8 



5539 

2401 
3138 



89.5 

68.7 
53.6 



4293 

2348 
1945 



5180 

2283 
2897 
3261 



4244 

2240 
2004 
1503 



2.5% 



*** 



0.4 



227* 

146** 
80 



_] ] ** 

-4.2*** 
-2.4*** 



-265*** 

-152*** 
-113*** 



439** 

216** 

223* 

277* 



-241 



*** 



-125*** 

-115*** 
-65 



4.0% 

5.4 
4.1 

1.1 



4.1% 

6.1 
2.6 



-1.3% 

-6.0 
-4.5 



-6.2% 

-6.5 
-5.8 



8.5% 

9.5 
7.7 
8.5 



-5.7% 

-5.6 
-5.7 
-4.3 
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Late cohort 

Average total earnings ($) 
Years 1-2 



5873 



5774 



99 



1.7% 



Year 1 
Year 2 



2584 
3289 



2473 
3301 



111 
-12 



4.5 
-0.4 



Average total AFDC 
payments received ($) 
Years 1-2 



4043 



4333 



-289* 



-6.7% 



Year 1 
Year 2 



2253 
1791 



2426 
1907 



- 1 73*** 
• 116*** 



-7.1 
-6.1 



Youngest child age 6 or 
older 

Average total earnings ($) 
Years 1-2 



6228 



5755 



473** 



8.2% 



Year 1 
Year 2 



2790 
3438 



2506 
3249 



284*** 
190 



11.3 
5.8 



Average total AFDC 
payments received ($) 
Years 1-2 



3720 



4024 



-304 



*** 



-7.5% 



Year 1 2053 

Year 2 1667 

Youngest child age 3-5 

Average total earnings ($) 

Years 1-2 5202 

Year 1 2268 

Year 2 2935 

Average total AFDC 
payments received ($) 

Years 1-2 4467 

Year 1 2405 

Year 2 2062 



2224 
1800 



5183 

2214 
2969 



4677 

2538 
2139 



-171 * ** 
•133*** 



20 



54 



-210*** 

-133*** 
-77* 



-7.7 
-7.4 



0.4% 

2.4 
-1.1 



-4.5% 

-5.2 
-3.6 



Statistical significance: *** = 1 percent; ** = 5 percent; * = 10 percent. 
Source:, Kemple, Friedlander and Fellerath (1995), Tables 3, 6.2, 6.4. 
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Table 13 

Effects of New Chance on Low-Income Mothers 





Experimentals 


Controls 


Difference 


Participation Rates 
Any education program 

Basic education/GED 

High school 

college 

Other education 


85.3% 
79.4 
2.5 

IZ.J 

20.1 


60.4% 
47.1 

3.6 

1 .y 
17.8 


24.9%*** 

32.4*** 

-1.1 

A 9fc 

2.3 


Skills training/unpaid work 
Skills training 
Unpaid work 


35.2 
33.3 
6.3 


23.3 
22.5 
2.2 


11.8*** 

10.8*** 
4 j*** 


Educational Outcomes 
Earned GED or high school diploma 
Earned GED 

Earned high school diploma 


43.1% 
36.8 

0.0 


30.0% 
21.1 

y.2 


13.1%*** 

15.8*** 

-2.0 


coiiieu uuue ceruiicdie or license 


J Z.D 


1 1 A 


A 1 
U. 1 


r^rpHitQ \r\\uc*rf\ A A c\r A H*»crr*»£» 




7 1 


z.o 


Average TABE reading score at 18 months 


748.7 


748.3 


0.4 


Fertilitv-related Outcomes 
Pregnancy 
Birth 
Abortion 


57.0% 

28.4 

14.9 


53.0% 

26.2 

11.1 


4.0%* 
2.2 " 
3.8* 


Sexually abstinent, not pregnant 


17.8 


19.5 


-1.7 


Sexually active, contracepting regularly 


37.0 


41.0 


-4.0* 


Sexually active, not contracepting regularly 


30.2 


25.2 


49** 


Emplovment Outcomes 
Ever employed, months 1-8 


42.6 


44.9 


-2.2 


Average number of weeks employed in 
months 1-18 


9.1 


10.8 


-1.8** 


Average earnings in months 1-18 


$1,366 


$1,708 


$-342** 


Receiving welfare at 18 months 


82.1% 


81.5% 


0.7% 


Average total income in the month prior to 
the follow-up interview 


$802 


$799 


%3 



Statistical significance: *** = 1 percent; ** = 5 percent; * = 10 percent. 
Source: Quint, Polit, Bos, and Cave (1994), Tables 3-6. 



Table 14 
Effects of JOBSTART 



Experimentals 



Controls 



Difference 



Received GED or high school 
diploma by end of year 4 

Full sample 

Men 

Custodial mothers 
Other women 

Ever employed (%) 
Years 1-4 
Year 1 
Year 2 
Year 3 
Year 4 

Total hours worked 
Years 1-4 
Year 1 
Year 2 
Year 3 
Year 4 

Total earnings 
Years 1-4 
Year 1 
Year 2 
Year 3 
Year 4 

Custodial Mothers 

Ever employed (%) 
Years 1-4 

Total Earnings ($) 
Years 1-4 

Other Women 

Ever employed (%) 
Years 1-4 

Total Earnings ($) 
Years 1-4 



42.0 
42.0 
42.0 
41.6 



86.4 
56.5 
71.0 
61.8 
65.7 



3031 
441 
760 
899 
930 



$17,010 
2097 
3991 
5329 
5592 



75.4% 
$8959 

84.3 
$13,923 



28.6 
28.3 
26.7 
31.3 



86.0 
60.8 
67.5 
61.5 
64.5 



3071 
550 
775 
855 
890 



$16,796 
2596 
4112 
4906 
5182 



71.0% 



$8334 



85.3 



$13,310 



13 4 *** 

13/7 *** 

15.3 *** 

10.4 ** 



0.4 

-4.3** 
3.5* 
0.3 
1.3 



-40 

-109*** 
-15 

44 

40 



$214 
_499*** 

-121 
423 
410 



4.5% 
$625 

-1.0 
$613 



I 

■ Men 
Ever employed (%) 
Years 1-4 " 94.1 94.5 -0.4 

i 

B Custodial Mothers 
Ever received AFDC (%) 

Years 1-4 84.8 81.6 3.2 



1 
1 



I 
I 
I 
I 

E 
I 
I 
I 



Total Earnings ($) 

Years 1-4 23,364 23,637 -273 



Total AFDC income ($) 

Years 1-4 9371 9334 37 

Ever pregnant (%) 

Years 1-4 76.1 67.5 8.6** 

8 Ever gave birth (%) 
Years 1-4 67.8 57.9 9.9** 

S Other Women 
Ever received AFDC (%) 
Years 1-4 



38.0 45.1 -7.1 



Total AFDC income ($) 

Years 1-4 3204 3979 -775 



I 

■ Ever pregnant (%) 

Years 1-4 64.4 65.6 -1.2 

Ever gave birth (%) 

Years 1-4 52.7 56.5 -3.9 

Men Ever arrested 

Yearl 35.1 35.1 -0.1 

Years 1-4 68.9 74.8 -5.8 

Ever used any drug in year 4 25.4 31.0 -5.5 

Ever used any drug excluding 3.7 10.5 -6. 9* 

marijuana in year 4 

Ever used marijuana in year 4 25.3 30.2 -4.9 



Statistical significance: *** ~ 1 percent; ** = 5 percent; * = 10 percent. 
Source: Cave, Bos, Doolittle, and Toussaint (1993), Tables 2—7. 
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Table 15 

Effects on Quarterly Earnings for Major Subgroups of AFDC Applicants and 
Recipients, for Various Welfare-to-work Programs 



Subgroup 


San Diego 


Baltimore 


Virginia 


Arkansas 


County 


First Tier 












Applicants with No 
Prior AFDC 


$+37 


$+121 


$-13 


$+26 


$— 


Second Tier 












Arvnlirant T?*»tiirn^i»c 


4.1 SR** 


, 100*** 

T 1 OO 


4-1 14* 

TI It 


4.0 1 1 * * * 




Applicant Returnees with 
Less than $3000 Prior 
Earnings 


i 1 C 1 * * 

4* 1 j 1 


+Zj3 


+20 


+202** 




l mra l ier 












All Rp/Mtv^ntc 
r\xi ixCCipiCIlla 




+ j 1 




, in 

+ [y 


+4o 


Recipients with More than 
Two Years on AFDC 




-0 


+1 10** 


4-14 

Tit 




Recipients with No 
Prior Earnings 




+104** 


+70 


+29 


+12 


Recipients with No Prior 
Earnings and More than 
Two Years on AFDC 




+88 


+94* 


+28 




All AFDC 












Quarterly Earnings Impact 


+118** 




+72** 


+70** 


+ 19 


Average Control-Group 
Earnings 


773 


634 


541 


257 


451 



Statistical significance: * = 10 percent; ** = 5 percent, *** = 1 percent. 
Source: Friedlander (1988), Table 1. 
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Table 16 

Effects of JTPA, by Type of Service 



Impact per enrollee 

Mean earnings 

, of enrollees In dollars As a percent 

Adult women 

Classroom training $12,008 $630 5,5% 

OJT/JSA 17,319 2,292 153 

Other services 14,191 3,949 38.6 

Adult men 

Classroom training 19,349 $1,287 7.1% 

OJT/JSA 23,621 2,109 9.8 

Other services 20,023 941 4.9 

Female youths 

Classroom training $ 10,279 $839 8.9% 

OJT/JSA 14,256 -578 -3.9 

Other services $8,286 ,33 _o.4 

Male youth 
non-arrestees 

Classroom training $ 1 6,362 

OJT/JSA 21,101 

Other services 12,819 

Statistical significance: *** = 1 percent 
Source: Bloom et ah (1994), Exhibits 1 



$251 1.6% 
-3,012 -12.5 

-438 -3.3 

; ** = 5 percent; * = 10 percent. 
4 and 15. 
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Table 17 

Effect of JTPA on Earnings, by Target Group and Follow-up Period 



Impact per enrollee 



Mean earnings 
of enrollees 



In 
dollars 



As a 
percent 



Adult women 

Months 1-6 

Months 7-18 

Months 19-30 

Total 
Adult men 

Months 1-6 

Months 7-18 

Months 19-30 

Total 
Fen-ale youths 

Months 1-6 

Months 7-18 

Months 19-30 

Total 

Male youth non-arrestees 
Months 1-6 
Months 7-18 
Months 19-30 
Total 



$2,138 
5,794 
6,292 

14,224 

$3,718 
8,807 
8,996 

21,521 

$1,564 
4,199 
4,744 

10,508 

$2,628 
6,538 
7,252 

16,418 



$170* 
820*** 

1,837*** 

$204 

538 

856** 
1,599* 

$-5 
53 
162 
210 

$61 
-289 
-639 
-868 



8.6% 
16.5 
15.6 
14.8 

5.8% 
6.5 
10.5 
8.0 

-0.3% 
1.3 
3.5 
2.0 

2.4% 
-4.2 
-8.1 
-5.0 



Statistical significance: *** = 1 percent; ** = 5 percent; * = 10 percent. 
Source: Bloom et al. (1994), Exhibit 6. 
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Table 20 

Estimated Net Present Value per Corpsmember (1977 Dollars) 



Society 



Corpsmembers Rest of Society 



Benefits 

Output produced by $4653 
Corpsmembers 

Reduced dependence on $ 1 58 

transfer programs 

Reduced criminal activity $2112 

Reduced drug/alcohol use $30 

Utilization of alternative $390 
services 

Other benefits + 

Total benefits $7343 

Costs 

Program operating $ 1 449 

expenditures 

Opportunity cost of $88 1 

Corpsmember labor 

Unbudgeted expenditures $46 

other than Corpsmember 

labor 

Total costs $5070 



Net present value $2271 
(benefits less costs) 

Benefit-cost ratio 1.45 



$3397 

-$1357 

-$169 
$0 
-$49 

+ 

$1823 

-$1208 

$728 

-$185 

-$665 

$2485 

1.82 



$1255 

$1515 

$2281 

$30 

$439 

+ 

$5520 

$5351 

$153 

$231 

$5736 

-$214 

0.96 



Source: Long, Mallar, and Thornton ( 198 1 ), Table 6. 
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Table 22 

Benefits and Costs of JTPA Per Enrollee, by Target Group 





Enrollees 


All others 


Society 




Adult women 






Earnings gain (minus OJT subsidy) 


$1,703 


$-- 


$1,707 


Training cost 


56 


-1,227 


-1,171 


Welfare benefit reduction 


-235 


235 




OJT wage subsidy 


154 


-154 




Net benefits 


1,678 


-1,146 


532 




Adult men 






Earnings gain (minus OJT subsidy) 


$1,401 


$-- 


$1,401 


Training cost 


100 


-931 


-831 


Welfare benefit reduction 


334 


-334 




OJT wage subsidy 


244 


-244 




Net benefits 


2,079 


-1,509 


570 




Female youths 






Earnings gain (minus OJT subsidy) 


$146 


$-- 


$146 


Training cost 


76 


-1,392 


-1,316 


Welfare benefit reduction 


-379 


379 




OJT wage subsidy 


74 


-74 




Net benefits 


83 


-1,087 


-1,170 




Male youth 








non-arrestees 






Earnings gain (minus OJT subsidy) 


$-949 


$-- 


$-949 


Training cost 


110 


-2,065 


-1,955 


Welfare benefit reduction 


119 


-119 




OJT wage subsidy 


100 


-100 




Net benefits 


-620 


-2,284 


-2,904 



Source: Bloom et al. (1994), Exhibit 16. 



Table 23 

Benefit-Cost Analysis of GAIN, 1993 Dollars 



Net present value of benefits minus costs for: 



All single parents 



Welfare 
Recipients 



923 



Government 
Budgets 



-833 



Taxpayers 



-990 



Society 
(taxpayers & 
recipients) 



-67 



Single parents not 
needing basic 
education 



2340 



-622 



-824 



1516 



Single parents needing 
basic education 

Heads of 2-parent 
families 

Single parents in: 

Alameda 
Butte 

Los Angeles 
Riverside 
San Diego 
Tulane 



-117 



-186 



1090 
1585 
■1561 
1900 
948 
1577 



-391 



-607 



-3054 
54 
-3442 
2936 
767 
-2261 



-530 



-652 



-3193 
-133 

-3485 
2559 
702 

-2396 



-647 



-838 



-2103 
1452 

-5046 
4458 
1649 
-819 



Source: Riccio, Friedlander, and Freedman (1994), Table 7.7, 7.10. 



Table 24 



Five-year estimated net gains and losses and returns per member, 

Project Independence 

(1993 dollars) 







Accounting Perspective: 




Component of Analysis: 


Welfare 
Sample 


Government 
Budget 


Taxpayer 


Society 


Earnings 


600 


0 


0 


600 


Fringe benefits 


89 


0 


0 


89 


Tax payments 
Payroll taxes 
Income and sales taxes 


-46 

2j 


92 
-25 


46 
-25 


0 
U 


Tranfer programs 
AFDC payments 
Food Stamps 
UI compensation 
xotai Mecicaia 

Transfer administrative costs 


-All 

-209 
24 

-43U 

0 


422 
209 
-24 
430 
118 


422 
209 
-24 
430 
118 


c\ 

u 
0 
0 
u 

118 


Net cost of Project 
Independence and non-Project 
Independence activities and 


n 


-1150 


-1150 


-1 1 so 


Preference for work over 
welfare 


+ 


0 


+ 


+ 


Foregone personal and family 
activities 




0 


0 




Value of education not reflected 
in earnings 


+ 


0 


+ 


+ 


Net gain or loss 
(net present value) 


-369 


72 


26 


-343 


Return to government budget 
per net dollar invested in 
Project Independence and non- 
Project Independence activities 
and services 


n/a 


1.06 per $1 


1.02 per $1 


n/a 



Source: Kemple, Friedlander, and Pellerath (1995), Table 7.5. 
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Table 25 

Benefit-cost analysis of the JOBSTART program 



Accounting Perspective 



Component 



Program 
Participants 



Taxpayers 



Society 



Increased earnings and fringe benefits 

Increased tax payments 
Payroll taxes 
Income and sales taxes 

Reduced use of transfer programs 
AFDC payments 
Food Stamp payments 
General Assistance payments 
Payments from other public 

programs 
AFDC administrative costs 
Food- Stamp administrative costs 

Reduced use of community education 
and training programs 

Reduced criminal activity and income 

JOBSTART operating costs 

Compensation for program-related 
expenses 

Additional support services 

Change in family's financial needs 
owing to program effects on 
childbearing 

Value of education not reflected in 
earnings 

Preference for work over welfare 
Forgone leisure time and activities 



$69 



$74 

-$34 

$28 



0 
0 

0 

$0 
$0 

$117 

? 



$0 



+ 
+ 



-$74 
$34 
-$28 
+ 

+ 
+ 



+ 

-$3863 
-$568 

-$117 

9 



$69 



0 
0 



$0 
$0 
$0 
0 

+ 
+ 



+ 

-$3863 
-$568 

$0 

9 



0 



Total 



$254 



-$4540 



-$4286 



All costs are in 1986 dollars Components that have not been assigned a dollar value are shown * 
a benefit (+), cost (-), or neither a benefit or a cost (0). 



as 



Source: Cave, Bos, Doolittle, and Toussaint (1993), Table 7.9. 
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